Steam review bombing is working, and Chinese players are a powerful new voice

In early April, just a month and a half after it released on consoles and two weeks after it launched on Steam, action-RPG Nier: Automata passed a million sales. Despite a few performance issues on PC, Nier launched to acclaim and racked up thousands of positive reviews. It was the surprise hit of the year until PlayerUnknown's Battlegrounds, and a wonderful success for quirky director Yoko Taro (whose previous few games combined likely sold less than Automata did in a month). And then, on April 28, this happened:

1,113 negative reviews slammed Nier's Steam page in a single day, essentially doubling the total number of negative reviews overnight. That's not enough to sink a game like Nier, but it is enough to drop its ‘recent reviews’ from positive to ‘mixed’ in early May. What happened? Why the sudden rush of negativity? On April 27, Square Enix released Nier: Automata in Asia—without Chinese language support. And hours after release, they doubled the price in China. It turns out that's a really, really good way to piss off thousands of players.

Review bombing isn't a new on Steam; we first saw the power of a flood of negative reviews back in 2015, when Valve briefly tried to roll out paid mods for The Elder Scrolls V: Skyrim. But the reaction to Nier: Automata is indicative of changes in the power structure of the Steam community. 

By users, China is now Steam's third largest country, with around 17 million, behind only the United States and Russia. But by bandwidth consumed, China is actually in second place, swallowing up nearly 10 percent of Steam's total traffic (the US accounts for 19.7 percent, Russia only 5.8 percent). It's a huge audience, and it's growing quickly. This presents PC developers a huge opportunity to sell games to millions of players they couldn't reach just a few years ago. But that opportunity comes with new risks, too.

Tried-and-true review bombing

Even without tapping into a whole new audience in China, developers in 2017 have to keep in mind the possibility that their game could be review bombed for any number of reasons. Maybe a political or social issue will draw an angry mob; maybe gamers will voice complaints about a glaring performance issue or a price they consider unfair.

We've seen two high profile cases of this in just the last month. Take a look at the recent reviews for Grand Theft Auto 5 and Crusader Kings 2. 

If you've been following the news, you'll recall GTA5 publisher Take-Two decided to shut down popular mod tool OpenIV, racking up about 35,000 negative reviews in a few days. Crusader Kings 2 accrued just a few hundred negative reviews over Paradox's decision to raise the prices of its games in some regions. In both cases, Rockstar and Paradox backpedaled after an overwhelmingly negative response.

Did those negative reviews actually affect sales of either game? Hard to say for sure, but probably not—both have already sold loads of copies. "On the per-game basis, it seems that review bombing doesn't hurt short-term sales much," writes Sergey Galyonkin, creator of SteamSpy. "Payday 2 sold more in the month after the [2015 microtransaction] controversy than in the month before, but they had a pretty big discount :) And Ark sold twice as more in the month after the review bombing [over its expansion] than in the month before despite having a smaller discount during that period and no F2P event.  Of course, this kind of reaction is usually because of the diminished players' trust in the game, but it's something that is hard to measure."

As Galyonkin says, damage to community goodwill is likely the bigger concern. And directly commenting on the actions of an angry fanbase is a touchy subject; unsurprisingly, the developers and publishers I reached out to for comment who have experienced Steam review bombing either declined to talk or did not respond.

For many gamers, leaving a negative Steam review is the only actionable way to make their voices heard. It's more direct, and likely more effective, than tweeting or posting on a message board. The system may be abusable, but in some cases it's working as intended: players are simply commenting on a game and giving prospective buyers valuable information. And Valve has taken steps to limit some potential issues with the system by removing reviews tied to free copies from a game's overall score.

Another change to Steam has also had a large positive impact: refunds. "Refunds are so great," says Lars Doucet, the developer of tower defense game Defender's Quest. "Now that they have refunds, if someone hates my game, they're going to ask for a refund, and usually the grumpiness that they would have directed at a review gets shouted into the refund system instead. Now I don't have an angry player, and the angry player gets their money back and isn't angry any more and probably doesn't feel like they have to post a nasty review (some will still do it, but not most). I have a separate channel for reading messages from people who wanted refunds, which is now where most of the really nasty comments go. That's fine. They can stay there. This system doesn't solve everything but it was a major plus for both developers and players in my eyes."

Doucet frequently blogs about Steam on a variety of topics, offering up some great data analysis and savvy commentary on Steam's Discovery system, reviews, the Controller, and more. Though Defender's Quest has a 97 percent positive rating on Steam, I thought Doucet would have some insight into review bombing, generally. He pointed out one issue he's written about before, with suggested improvements to how Valve sorts games based on user ratings. If Valve did improve its sorting, that could, potentially, change the visibility of some games on Steam with fewer reviews but higher positive percentages. And that could, in turn, make review bombing a more powerful tool.

I also asked if he thought review bombing was a result of the review system working as intended, or if there's a better way for players to make their voices heard.

"Obviously as a developer the idea of being flooded with a torrent of hate over something players didn't like makes me super scared, especially when it's not something 'merely' personal like your twitter feed, it's my store page which has a direct connection to my livelihood," he says. "However, it's hard for me to feel too sorry for a big AAA company that does something obviously tone deaf and anti-consumer and pays a public price for it. At the same time, if you're a small developer and you catch that storm you're fielding that red hot fire with your own face, and these mobs can get really nasty. The one limiting factor here is, unlike in a Twitter mob, the only person who has a right to leave a review is someone who's paid for your game [on Steam]."

Doucet said that there's a trade-off with that new requirement, since developers who work to build an audience outside of Steam, through a Kickstarter, for example, lose out on the value of those positive reviews. "On balance, I will make the trade—I prefer being slightly less vulnerable to random hate mobs who just happen to have bundle keys for my game over boosting my user reviews with crowdfunding keys. Others feel differently, of course. I think if they smoothed out the hard thresholds using the Wilson score method [mentioned in his article on reviews], this tension wouldn't be as sharp."

Doucet has also written about the huge opportunities for indie developers in the Chinese market, which brings us back to the original topic. On top of these concerns, which have been building on Steam over the past few years, developers now have to consider the dangers of releasing a game in China without localization.

China is now too big to ignore

In October 2016, Chinese players filed hundreds of negative reviews against Football Manager 2017 protesting the lack of a Chinese localization. Some of the anger stemmed from comments made by the studio's director five years earlier, saying more sales in China would justify a localization. More anger came from the developer's decision to make community translations part of the game's Steam Workshop (it officially supported 16 languages at launch).

Publisher Sega stepped in after the outcry and announced a Chinese translation, which was released in April. On SteamSpy's review chart, you can see the jump in positive reviews and a decrease in negative reviews as some players reversed their earlier thumb down.

Still, many of the negative reviews remained; Football Manager 2017 still has a 44 percent positive overall rating on Steam. And the negative reviews didn't just affect FM2017; on the same days it was review bombed, so was Football Manager 2016.

Several Square Enix games suffered collateral damage after Nier: Automata's pricing debacle. Chinese players also review bombed Rise of the Tomb Raider on April 28, giving it negative reviews to punish Square Enix for Automata's price change in Chinese currency. It was the only significant spike in negative reviews after Rise of the Tomb Raider's first week of release.

Even more recently, Darkest Dungeon developer Red Hook launched the Crimson Court expansion and saw a smaller wave of criticism from a Chinese playerbase tired of waiting for a localization that had been talked about as far back as 2015 and still not materialized. (Darkest Dungeon has had a community-driven Chinese localization in progress for months, but it's not yet finished).

As always, it's hard to draw any real correlation between these negative reviews and a tangible impact on sales. It's a lesson all developers will soon have to take note of: If Chinese players are promised a localization, they expect it, and Steam reviews are how they’ll let you know.

But with localization, Steam's rise in China could also be a new lifeline for smaller developers struggling to make a profit amid the dozens of games hitting Steam each day. Getting the pricing right for the region and properly localizing a game can be a huge windfall. As Doucet wrote about releasing games in China earlier this year: 

"I have never had a localization pay for itself this quickly, not to mention this unambiguously… Of all the revenue Defender's Quest has earned from China on Steam in its entire lifetime, 45% of it was earned last week. That's right, we basically doubled our lifetime sales from China almost immediately!"

Of course, it's not quite that simple—there's no guarantee every game will be a smash hit in China, and there's not even a guarantee that Steam will even remain available in China. Games are routinely banned in China, but so far Steam has escaped government regulation, possibly due to Dota 2's popularity. Megacorporation Tencent is re-launching a Steam competitor in China in July, which could massively upset the PC market there. The only real constant here is how fast things are changing.

But as long as Steam exists in China, expect the Chinese audience's influence to grow. 17 million players and counting are hard to ignore.

Data gathered from SteamSpy.Thanks to Sergey Galyonkin for an invaluable tool.