Another Rant About Beta Testing Video Games

As I’ve been trying to make my job less mentally taxing and find ways to reinvest myself in the work I’m doing that actually pays bills, I’ve been thinking a lot about what it means to test software and what it means to play video games in beta or early access. I’m not going to go on another full rant about it, not so soon after the last one, anyway, but given how much of my life is given to testing software and how much is given to video games, it’s difficult to avoid considering the place those two things intersect.

Continue reading

I’m Just Going To Rant A Bit

I don’t talk about my day job in any specifics too frequently, but I test software for a living. Techincally software and hardware, but I focus mainly on software and the proprietary hardware said software runs on. I’ve done electrical testing, mechanical testing, software testing, and, my personal favorite, destructive testing. While the specifics vary from project to project, each type of testing is pretty much exactly what it sounds like. As of this writing, I’ve been in the industry for almost eight years and have pretty much reached a point where I have the skills required to tackle just about any project.

What all of this means is that I have a pretty good grasp of the testing effort that goes into software production and, as part of keeping up on the industry I work in, I have a pretty good idea of what testing all software, except the most proprietary and niche stuff, will look like. So when I say the testing and release for most triple AAA games is bullshit, I mean it and you can trust, at least a little bit, that I know what I’m talking about.

I mean, have you ever looked at the credits for a game and noticed how few QA and testers (the job name changes from company to company since there’s nothing to really enforce consistency across the various industries that employ software testers) there are in comparison to developers? In my industry, it is pretty much required to have at least one tester to every one or two software developers. The best bet is to have a one-to-one ratio since it can otherwise force testers to work extra hours to make up the difference or to cut corners in order to finish on the required deadline. Because let me tell you, testing is the first thing to be given less time to work when there’s a looming deadline and the developers need more time to work.

It is the easiest to do that in industries where no one’s life is at stake. I work for a company that produces a lot of different types of software and products that use that software, so while my testing has to be done with the thought in mind that the worst case scenario can involve bodily harm or even death, that’s not really a concern for most of my coworkers. Which is why my team has three testers to work with four and a half (one outside consultant who works part time) developers. We’re a bit short staffed, since those three testers are also responsible for working with the electrical and mechanical engineers as well as the software developers, but we just need a fourth person to keep up with the work that needs doing and maintaining records, test equipment, etc. We’re not falling behind (yet).

So when you see a dozen or so testers at the end of a video game’s credits, following a few hundred software developers, I think you can start to see why so new games seem to be getting buggier and buggier as time goes on.

Like most problems, this one is also multi-faceted. Developers who have the opportunity to rest do better work. Developers allowed to work on a single piece of the project, start to finish, do better work. The current methods of just throwing more bodies at problems and expecting the work to get done faster ignores the limits of human consciousness and just how much time gets wasted by bringining someone up to speed so they can peck away at a problem for a day or two before it gets handed off to someone else.

Not every studio does that, of course, or else the industry probably would have collapsed by now. But as work gets spread out and testing employees get cut infavor of customer betas or alphas and the work of actual testing is placed on the shoulders of people who preordered the game and have no knowledge of how the software works (or how testing should happen), quality goes down. More and more games, as a result of pre-sales, are putting testing work on their customers and trying to frame getting people to pay them to work on the game as an incentive or bonus for paying sometimes years ahead of the game’s release (and who knows how long after that it’ll be actually playable).

As someone who works in the industry, I find the practice abhorrent and kind of insulting. It takes a lot of work and skill to be good at finding, investigating, writing up bugs. I spend 40-50 hours a week doing that, most months, and now it is not only culturally acceptable for companies to expect me to pay for the privilege to do that for their games, but I get people telling me that they’d make great software testers just because they’ve played a lot of games on Steam prior to their actual release. I’ve even had a few tell me my job isn’t demanding or tiring because it must be so easy since anyone can do it.

Saying anyone can test software is like saying anyone can write. Sure, that is technically true, but there’s a pretty wide gulf of practice, experience, and skill between testing and doing it professionally. And if you rely on people who have no professional skills to do all your testing, your product is going to fall apart the instant it reaches a wider audience. Turning game pre-orders into Betas and Alphas is one of the worst things to happen to my industry, let alone my hobby, and I’m so sick of 2-4 weeks of bugs followed by 1-2 months of patching all the things fixing those bugs fucked up becoming the industry norm.

Which, of course, isn’t to say that user feedback or end-user testing isn’t a valuable tool in the development process. It is an incredibly valuable tool, since there’s testing that can’t even be done without end users (stress testing is a big one that frequently comes up in the video game industry), but it shouldn’t be a testing solution.

Rant over. Stop paying to do the work companies don’t want to pay people to do. Don’t preorder games. Any game scarcity at this point is manufactured thanks to digital downloads and they’re just trying to offload costs more than ever so they can throw 500 developers at a triple-A game only for it to suck until the community finds and points out all the problems to them. Nothing’s going to change so long as people keep this up.

The Reassuring Failure of Spotify’s Discover Weekly

First: a follow-up. I wrote yesterday about struggling to obtain an item in Skyward Sword that I need for gear upgrades. During that evening’s video game time, I played Skyward Sword again and killed three enemies that dropped this item and every single one dropped it. Three in a row, right there, staring at me. And completely inaccessible. No way to get them, even a little bit. 100% impossible. Absolutely devastating.

Continue reading