In the past, PC Gamer has interviewed developers about what it's like to release a game on Steam as a small team, and written about what we think Valve needs to do to make Steam a better store for PC gamers. We're publishing the results of this developer survey to give you more insight into the business of releasing games on Steam, and how developers feel about the biggest PC game distributor.
For every game sold on Steam, Valve gets a cut. Today, according to a survey of more than 200 PC developers, only 39 percent believe Valve is earning its share, compared to 59 percent at some point in the past. This is that survey.
Operation Tell Valve All The Things
My name’s Lars Doucet, I’m one of the developers of the indie game Defender’s Quest: Valley of the Forgotten, I blog regularly on my site, Gamasutra, and as of today, PC Gamer. I have a lot of opinions about Steam. So do many other developers. Whether it’s Steam Direct, review bombs, forum policy, “fake games,” or gripes about revenue share, everybody has a bone to pick with Valve, and everybody has a different angle.
However, a tangled mess of opinions is neither a useful signal for Valve nor a coherent rallying cry for those who seek change. What’s true for woodworking is true for knife fighting and for criticism—take the measure of your target twice before the first cut.
To that end, my developer friends and I started Operation Tell Valve All The Things—a recurring survey of people who publish games on Steam. Developers raised issues and voted on them, which I processed into a report for Valve. We first did this in 2016, and have just now completed the 2017 edition. Valve got its copy two weeks ago, and now we’re releasing the public report on PC Gamer.
Taken together, we can see that many (but not all) developers are unhappy with Valve, and this is largely in response to recent developments.
You can read the full report here, and a cool interactive infographic is available here, courtesy of my friend Jason O’Neil. This article is my breakdown of what we found, highlighting the biggest topics—the number of games being released on Steam, Valve's reliance on algorithms, and how hard it is for devs to get the word out about their games—to offer insight into the state of Steam and PC gaming in 2017.
In all, 232 developers participated in the survey, of which 223 opted to share their responses publicly. Scraping the Steam API suggests there are roughly 10,000 registered developers on Steam, and Valve representative Alden Kroll told me in a recent email that approximately 8,200 of those have ever earned any money. Either way, this gives us a sample size of just over 2 percent. It’s a fairly large sample, but it’s hardly random as those surveyed came mostly from my own network of contacts.
Before we dig in, a few disclosures. I’m a developer with a game published on Steam, I’ve appeared on stage at Steam Dev Days to discuss the Steam Controller API, and I’m currently under freelance contract with Valve documenting the Steam Controller and its API. Also, a few months back I was invited to Valve headquarters for an interview, though I ultimately didn’t get an offer. I was not paid by Valve (or anyone else) to conduct Operation Tell Valve All The Things, though as the public report neared completion PC Gamer approached me about publishing a presentation article for it (the piece you’re reading now, which they are paying me for). To be sure, I would have released the public report with or without interest from a publication. As for my personal opinions, there are some things I really like about Valve and Steam, and some things I really don’t like, but I’m going to do my best to leave my personal feelings out of this and just let the developers’ collective voices speak for themselves.
The big picture
Now let's get back to this graph about whether Steam is earning its cut:
I also asked how developers feel about Valve as a company:
Taken together, we can see that many (but not all) developers are unhappy with Valve, and this is largely in response to recent developments. It was easier to earn money on Steam years ago, but it was also much harder to get onto the store at all. Now, the floodgates have opened and just about anyone can publish their game on Steam. Steam will likely pass 6,000 game releases in 2017 alone, or more than 15 per day.
As one developer puts it bluntly:
With everything being automated or farmed out to users, and even the best and highly promoted independent games falling through the cracks, Steam's last few advantages are eroding—no longer is there any sort of curation or much assistance on anything, and the huge audience doesn't matter if they don't buy our games. Steam taking [Valve's cut of the revenue] of every sale in order to... not promote games? Have users do the work? Support racist, sexist, hate-filled forums that glorify abuse and allow players to hold developers hostage by rating games poorly? I don't see the value proposition.
However, another says:
It's pretty clear Valve is able to drive traffic and sales to my game at a far greater rate than I could on my own, so I'd rather they take [Valve's cut of the revenue] of a significant amount than me taking 100% of very little.
We asked developers which platform they would prefer be PC gaming's market leader. The question specifically added—"Wave a magic wand and tomorrow this platform has the same size and power that Steam has right now:"
More than a quarter of developers would like to see the crown go to Itch.io, but a clear majority still wants Steam on top.
We also asked developers to rank, in order, the biggest perceived threats to their success.
The biggest threat to our surveyed developers was how hard it is to get noticed by the nexus of press, YouTubers, streamers, and other tastemakers seen as driving the sales of the most successful games.
Many developers are clearly worried about the flood of "low quality" games and "fake games" drowning out more earnest efforts, but despite all the attention that's gotten in the press and online conversations, it still ranks below competition from “high quality” games. In many developers' minds, there's simply a greater number of really good games these days than ever before, and everyone has to up their efforts. Concerns about picking an appropriate price or getting enough feedback before release ranked much lower.
Some choice comments:
It isn't that I dislike Steam, just that where once they drove in a lot of traffic via a curated storefront, carefully presented sales, popular events, etc, now they're just... a storefront. We're expected to drive our own traffic in. That isn't bad, but is it worth [Valve's cut of the revenue]? Only in "we control the marketshare, so you pay our rates for access to these customers" toll terms.
__________
Valve is pretty good (and improving) at being transparent, and addressing needs eventually, but has historically been hard to reach for urgent dev matters. Eg, launch day issues (Except finance, who are 100% present and on the ball.) Also, I have a more personal relationship with GOG and Humble staff. There are agents at each who keep in touch with me, and I feel our partnership is mutually beneficial.
__________
Power laws result in a few winners and a whole bunch of losers. A totally hands off "Let the market decision" approach means the rich get richer with fewer bigger hits. This is a poisonous ideology because these markets stagnate.
__________
I don't agree with every policy you [Valve] hold or action you've taken, but I understand that the position you're in is a fragile one and there is no one-size-fits-all solution. You seem to be trying to make the best of what you can, and I appreciate that, regardless of how effective the results are
__________
Valve needs an Amazon-like ability to recommend less popular games to users (current [recommendation algorithm] seems to be pretty terrible), otherwise they're going to continue to see this iOS Appstore-like race to the bottom where 1% of games occupy the spotlight and the other 99% languish with no visibility.
__________
The storefront tools already seem great to me, especially compared to previous platforms.
__________
Most of Steam's issues are structural. How do we get more sustainable developers? They are not about some little feature stuck in a menu somewhere.
__________
I think Valve are doing a pretty good job overall, but I'm biased because my niche indie game is doing very well on Steam!
__________
Discovery is a massive problem and understanding how, where and why traffic is routing to store pages is a mystery.
(For more comments, see Appendix H of the report)
The big picture:
- Trust in Steam has eroded somewhat.
- Many developers want an alternative to Steam but still see it as their best shot.
- Many developers aren't sure Valve's value proposition is worth the cut it takes.
- Despite the above, there's still a wide split in opinion—many are happy with Steam.
So what do the surveyed developers agree should be done about all this?
Top 10 issues
Developers raised and voted on 71 separate issues, and it was my job to distill that down to a top ten list. Developers voted twice—once on whether they agreed with an issue, and once on how important they thought it was. Only high-consensus issues were eligible for the top ten list, ranked by their importance score.
This is to avoid ambiguous situations where an issue is uncontroversial but not necessarily pressing—for instance, the highest consensus issue, the only one with zero disagreement, was: "Provide devs with traffic breakdown per region." However, this issue scored merely average on importance; everyone thought it was a good idea, but not a top priority.
Because nine respondents opted out of sharing their responses publicly, I had to recalculate the results for this report, which are slightly different from the list in Valve's. The difference is small—9 out of 10 issues are the same, and the top three issues ranked identically.
Each issue's description text was provided by the developer who raised the issue, with some occasional editing from me. For more information, see the report's Executive Summary or Appendix B: Issues.
1. Remove ability for users to delete developers’ comments on their reviews.
“We have users lie about the content/value of our game, then delete our comments on their review when we politely clarify for them and other users. It's absolutely insane that this is allowed.”
2. Clarify what a developer needs to do in order to qualify for various featuring opportunities.
“What does a game need to be eligible for a Daily Deal? Copies sold? Total revenue? Review score? Average play time? What about a free to play weekend? Getting banner featuring on the front page? Getting a slot on the popups when you launch Steam? Who would I even talk to? Steamworks forums? Contact Steam support? A specific email?”
3. Bug reporting/technical support should be built into Steam/Steamworks. Filter issues that are Steam issues to Steam support team instead of to the developer.
“Basically, it just gets really tiring to keep trying to convince users that "game won't download," "no file permissions," "steamAPI_init failed," is something they need to reach out to Steam for support with and not me. Having a tech request form and ticketing system built into Steam could really help ease the load on post-launch support.”
4. Devs should be able to have one unified landing page for all their games.
“It is a major pain to deal with four different community forums. We could do with being able to merge all our forums into a single developer forum.”
5. The Steam Community feature needs better ways to deal with toxic users.
“There's a lot of toxic users on our Steam community pages and we need better ways to appropriately deal with them. Even if we shut them down on our own pages, they still go on to harass and troll other developers as there's few consequences for their actions and no large coordinated moderating efforts, and their numbers are growing by the day.”
6. Valve should increase the overall volume of sustainable middle class developers.
“Create a report that shows how many teams are making over a million bucks a year. It doesn't need to name names, but it can mention what Valve is doing to help make Steam worth developing for. Improve these metrics year over year. A robust population of mid-tier developers means great, polished, innovative games keep being made. And those generate the new PC hits and fresh hit genres.”
7. Make reviews on the frontpage representative of the game's review percentage.
“Steam selects the 10 most relevant reviews and puts those on the frontpage. This is mostly based on upvotes. Negative reviews generally get most upvotes, because users hardly campaign for upvotes on positive reviews.
Example: At some point for [Game Title] we had 80% positive reviews but the frontpage showed 9 negative ones and 1 positive one. The "recent reviews" percentage at that point was 70%. Later we changed something and users changed those top ranking reviews and suddenly we had 10 positive reviews on the frontpage. The reviews on the frontpage are not representative of the general reception of the game at all, only of what vocal users are currently campaigning for. With 70% positive reviews there should be 7 positive reviews and 3 negative ones on the frontpage.”
8. The “Upcoming Games” list is becoming useless.
“The 'upcoming games' list used to be a major way to find out about new titles but there are now so many that it is becoming clogged. Not only is there lots of new stuff, but we often see a growing list of titles parked here week after week*. If the discovery feature is meant to work, it needs to gather some initial data about new games, otherwise it’s just blindly promoting things that came into the store with existing buzz.
*Slightly related: Some of these might be trying to milk free visibility by artificially adjusting their release date, polluting the list for everyone.”
9. Devs should have better tools and info for tracking and identifying players who abuse/hack multiplayer, bot/farm item drops, etc.
“Exactly what it says on the tin.”
10. Devs should have the possibility to send messages to forum users.
“Sometimes developers delete inappropriate topics from the Steam forums, but when they do that there's no way to send the user a message to explain why it was removed. Devs also can't edit topics (understandably), so if for example someone voices concern about a cheating tool by linking to it, then devs can't remove the link and their only option is to delete the topic, but then devs can't explain to the user why it was deleted."
Breakdown
The top ten issues were clustered in a few key topics, which shows where developers concerns mostly lie:
- 5 from Steam Community (#1, #3, #4, #9, #10)
- 1 from Steam Store (#2)
- 3 from Values (#5, #6, #8)
- 1 from User Ratings (#7)
In short, developers want Valve to improve community management, take greater strides to combat the "rich get richer" effect, and address various specific action items.
Bottom five issues
There was also a surprising sort of consensus in the opposite direction—issues certain developers raised that their peers voted strongly against, not only by disagreeing with the issue, but also by marking their disagreement as "very important." These were the least popular suggestions on the survey.
1. Allow devs to opt-out of reviews altogether
“Simply allow on a per-game basis to disable the Steam review system for that game. The control should be in the hands of the developer."
2. Do away with color coding the reviews
“Example: your review score is 69.9%. Now it’s gone from green to yellow and way fewer people are going to take a chance on it—just show the raw number. 69.9 Is not a low enough score to be actively warning people away.”
3. Set up a fund for video content
“It's no secret Valve wants to bulk up their video distribution platform—many developers have ties to animation and film production and would love to make high quality original content for Steam.”
4. Get rid of the % approval rating
“Count every review as positive—even if it is negative—and proudly display that a game has 'x' recommendations. Ensure Steam users understand that leaving a critique is implicit approval.”
5. Give curators a cut for every game purchased from their pages
“Curators are a great idea but are currently only done on a volunteer basis. Allowing them to monetize their efforts would shine a much bigger spotlight on their recommendations and drive whole new markets and serve niches.
"Allow curators to embed their own widgets on HTML that takes advantage of this. This would end up operating similar to a 'white label Steam' but with way less overhead or hassle; just maintain your own website of, say, 'Physics Based Visual Novels' and you could serve your community better there (and marketing there!) than fighting for [visibility] in the Steam ecosystem. This will be more valuable as more and more games launch on Steam.”
These were some of the results that surprised me the most, because the tallied votes differed sharply from initial conversations with developers raising issues; this might indicate these particular opinions belonged to a small but vocal minority.
Although many developers in the survey voiced concerns with the user rating system (as borne out by other favorably scored "User Rating" issues), as a group they are strongly opposed to any approach that essentially guts the current system. Also, the idea of Steam giving curators a referral fee or funding video content leaves most respondents cold.
Different strokes for different folks
The purpose of this report was to find out what developers can agree on, distill that into a prioritized list, and present it to Valve as a united front. That said, one shouldn't gloss over the many differences between developers, lest we ignore divergent and even conflicting interests in the name of solidarity.
For instance:
Developers who got in early have had a completely different experience than those who joined recently, and recent settlers now outnumber the pioneers.
Furthermore, let's not forget about money:
It gets even more interesting if you break it down into the six different communities I reached out to for my survey:
- Group A represents indie devs from all over the world, drawn from my contact list. This was one of the biggest groups.
- SteamWorks is the SteamWorks developer forum, open to any developer registered with Steam. This was the largest group.
- SteamProphet represents SteamProphet group Bravo and group ClarkTank, two mailing lists I run personally to discuss the business side of indie game development.
- Group D is another large group of indie developers from around the world.
- Groups B and C are two very small indie communities, mostly drawn from the UK and Canada, respectively. Both of these had only a handful of responses.
The best way to explore how different demographics voted on each issue is to use this visualization tool.
I should mention that a large number of surveyed developers opted out of sharing their demographic information publicly; therefore, the demographic report Valve received differs somewhat from the data presented here, to a greater extent than the very minor differences in issue scores. That said, the overall picture is still roughly the same.
Exploring the data
If you want to know more, I encourage you to read the full report.
I've included all the information necessary to reproduce my results, including my methodology (Appendix A), the original survey instruments (Appendices D and E), and the raw survey results themselves in spreadsheet form (Appendices F and G). I've double and triple-checked everything, but I'm not perfect; if anyone would like to check my work, feel free, and if you find a mistake, I will gladly update this article with corrections. You can also play with the data visualization tool embedded here. (Note: as of publication, a few of the groups in the community field in Survey 1 have been mislabeled, but all their survey data is correct).
Where we go from here
We presented this organized list of developers' priorities to Valve with the expectation of positive change. As of today, Valve has addressed 3 out of the 10 issues from last year's report (see Appendix B). It's a start, but we'd like to see more.
Ideally, by the time we start Operation Tell Valve All The Things v. 3.0, all our top ten issues will have been addressed, improving Steam reviews, Valve's approach to the community, and the number of developers who can sustainably create games for the PC.
