It would be nice to live in a world that makes sense, wouldn't it? One where you could easily reduce the issue of weapons balance to points on a spreadsheet, defeating the issue of game design with the raw power of maths. Unfortunately we don't live in that world, as backed up by Fallout: New Vegas' director Josh Sawyer—who recently dispensed a grain of wisdom: Balancing is "mostly vibes based".
This uncomfortable reminder was delivered via a recent YouTube video on the developer's channel (thanks, GamesRadar). Responding to a question he'd received on Tumblr, Sawyer gave a big shrug, stating that he doesn't "really believe in spending a huge amount of time, prior to people playing the game, in 'simulation land'. Because it doesn't really matter."
Sawyer recalls that, during development of Fallout: New Vegas, he'd set up a spreadsheet to tweak the game's weapons. However, it was more a case of sketching before drawing the rest of the game development owl: "I made a spreadsheet [that] listed out weapons by ammo type, and then I did some relative comparisons to damage threshold values … I used it for, maybe, a couple of months."
Sawyer says that the last time he accessed the doc was "March of 2010" which, as he says, was "well in advance of the game shipping." New Vegas released in October 2010, meaning the veteran dev was flying by the seat of his irradiated pants for over half a year.
"Ultimately, it's about the practical effect of it in the game," Sawyer argues. "You just play around with it." While he doesn't completely scrap the idea of making dedicated test levels to rinse the numbers, "most of what I did I would do within [actual] game levels. I would run around in the area around Vault 3, and then go into Vault 3 and test the weapons against the enemies in that area.
"To me that's really the important thing, to see how those things feel in practice."
This isn't exactly new wisdom. Earlier this year, Doom (2016)'s creative director Hugo Martin responded to a question about how the dev team balanced the destructive BFG. Simply put, they didn't: "it's a great gun, iconic to the franchise, but it is the 'kill everything button'."
While it's only weapons-adjacent, we've also seen this in practice with Helldivers 2, after recent tweaks to solo missions had players feeling overwhelmed by the new abundance of patrols. While Arrowhead Games points out that players were previously only getting 1/6th of the enemy regiments that a team of four would get, it likely didn't feel that way before the change due to other factors.
As Sawyer notes, "once you get player feedback, you have to listen to the player feedback", though he rightly points out that doesn't "necessarily mean acting on it."
Despite scrapping the spreadsheet eventually, Sawyer still sees it as a valuable part of the process: "Most of it is really about taking your best shot at a spreadsheet, laying it out, and then trying things out within the levels and seeing how it feels."
Now in fairness—Fallout: New Vegas is a singleplayer RPG, where the balance between various options matters less than making sure everything's viable at a baseline. If a player never stumbles into a broken combo then they're less likely to get mad because, as the old adage says, what you don't know can't hurt ya.
Weapons mouthfeel is still of vital importance in multiplayer games, but it'll more often fall to the wayside in favour of an even playing field. Finding out the level said field should be at, though, can be very difficult, like when a change to one of Destiny 2's guns proved controversial in 2022 because it was nerfed for raid reasons—though just as balancing itself is susceptible to vibes, so were the community's knee-jerk reactions. Sawyer's probably right, it's vibes all the way down.
