Valve continues to tinker with how reviews are sorted and displayed on Steam, with its latest change concerning "helpful reviews". Whether a review is helpful or not is determined by users, of course, and the metrics are pretty simple: the more "yes" clicks on a review, the more prominently displayed it will be. But naturally enough, that (on paper, easily manipulated) system has been manipulated.
"In a perfect world, people would truthfully mark a few reviews that were helpful for deciding to purchase or not purchase the game and we could use that data to directly determine the ten most helpful reviews," a new post on Steam reads. "Alas, it turns out that not everyone is as helpful as we would like. Instead, we are seeing more and more feedback from players that the helpful reviews shown on store pages aren't representative of how well people are actually enjoying the game."
Fancy people not always being helpful on a platform with multiple millions of users! Apparently, around 11 million people use the "helpful" rating system in a normal intended fashion, though there are "a small set of users on the far extreme" who systemically rate tonnes (ie, thousands) of reviews of the same game, presumably to help bias the most prominent reviews in favour of their own opinion.
"This behavior is not only humanly impossible, but definitely not a thoughtful indication of how 'helpful' each of those reviews were. These users also tend to rate up just the negative reviews while rating down the positive reviews (or vice-versa) in an attempt to distort which reviews are shown by default."
It continues: "Because of how many reviews these users are rating, they each have a disproportionate amount of influence over the display of helpful reviews and cause certain reviews to appear more prominently than they should be. This can result in a confusing appearance where the default set of reviews shown are negative, even when most players have posted positive reviews and clearly enjoy the game."
So what's Valve going to do? Firstly, users "outside the norm" (ie, those heavy raters who are clearly trying to fudge the system) will be treated differently when selecting whether a review is helpful or not. Secondly, the default "helpful" reviews shown most prominently – whether they be positive or negative – will be weighed in accordance to the overall score of the game. "For example, if the game is reviewed positively by 80% of reviewers, then the ten reviews shown by default on the store page will be 80% positive, showing eight positive and two negative."
You can read the whole post over here. The changes are available in the beta client right now.
