When a company says “we can’t afford a QA department”, what they’re really saying is, “we accept that our software will be infested with bugs, and quality is not important to us.” When they compound this basic error by saying, “the developers will just have to do their own QA”, they prove that they have no respect for developers or QA people, and you shouldn’t work for such a company in either capacity.
(Of course, a company like that isn’t about to hire any QA testers, so you folks haven’t got the option of working for them. And I’m not a QA tester, I’m a developer. So the rest of my advice is pretty much aimed at fellow devs — but that doesn’t mean I don’t respect you QA folks. Seriously, y’all deserve a lot more respect than you get, and I love it when you make my life easier by finding my bugs for me.)
The skills, talents, and basic mindset that make a good developer are entirely different from the ones that make a good QA person. Asking one to do the other’s job is a mistake as fundamental as expecting graphic designers and accountants to swap places. Let me explain:
Developers hate repetition. We hate having to repeat anything more than once or twice; that’s why some of us become developers in the first place: because we can write programs that automate repetitive drudgery, and hence banish it from our lives.
It’s not just repetitive tasks, either. A good developer will also have a strong aversion to repetitive code and data. If two procedures differ by only one or two minor details, a good developer will combine them into one, and parameterize it, or otherwise figure out a way for the machine to make the decision. (A poor coder, by contrast, will produce “copy-and-paste code” that’s scorned by more advanced devs, because it’s more brittle as well as being less elegant.) If data must be stored somewhere, the prevailing wisdom is to keep it in just one place — “Don’t Repeat Yourself” is the maxim.
But a good QA checker has to deal with repetition all the time. It’s a fundamental part of their job. Run this test plan — this particular series of actions — over and over again, on each of the platforms we need to test on. Log every step of it, so that if something does go wrong, we can reproduce it.
Then, once the coder has checked in the changes that (hopefully) will fix the bugs you found… do it all over again. Just the exact same way. On every platform.
This is a coder’s nightmare. A developer’s natural response to this situation is to write some kind of automated QA-testing software. Of course, that software itself will then have to be checked for bugs before you could possibly rely on it… and you can’t trust your software to find its own bugs.
In the absence of such software, asking a developer to do bug-checking means asking them to do something that goes against their very nature. You can force them to do it, but they’ll never be really good at it.
Of course, all of that is totally leaving aside the obvious point that asking someone to find the flaws in their own work means asking them to see past their own mental blind spots — a developer’s test plan will be based on the same assumptions as their code, without the coder even realizing it. And that’s to say nothing of the obvious implications once you put the human ego into the picture. Human beings are flawed, and too many see “find the bugs in your own code” as equivalent to “show me all the ways you suck”.
Don’t ask a developer to do QA. Development and QA testing are fundamentally different disciplines, with completely different skill-sets and personality requirements. A developer can no more cover for a QA tester than a QA tester could fill in for a developer.
And if you’re a developer at a company that wants you to do your own bug-checking? Your employer thinks what you do is so similar to bug-checking that you’ll be good at it. They think they can cut corners on quality, and pay you one salary to do two jobs at once (including one job that it is in your nature to hate). If you can’t make them see reason, it’s time to brush up your résumé.
3 Comments
Two points:
1) You probably saw this, recently:
@bramcohen “90% of coding is debugging. The other 10% is writing bugs”
2) Having been on both sides of the coding/QA dancefloor, I will note that the two things that QA likes least are: A) Code that is non-functional “out of the box,” so they can’t even START to get their jobs done, and B) Repetition.
(A) comes from code that hasn’t even been checked once for functionality after its been built and before it’s handed off to QA. In this respect, engineers SHOULD have a fundamental understanding of the QA process, because they should be used to running through their code themselves as they’re working on it, before saying “it’s ready” and handing it off to QA. This can include some combination of automated unit tests that exercise their code as an integral part of the build process.
(B) is why a large portion of QA engineering is devoted to working on automation of testing.
A really good QA tester is a highly inventive and creative person who finds new ways to break things. The whole point is that the end user can’t be counted on to only interact with software in a rigid, predetermined way. Good QA people are ones who can simulate the “infinite number of monkeys” that software is going to encounter once it gets out into the real world. These aren’t the kinds of people who get a thrill out of repetitive tasks. Though they acknowledge it can be part of the job (especially early on, before things are stable enough for automation), they’re going to be seeking ways to minimize it just as much.
Really, if you find someone who actually enjoys or even just doesn’t mind repetition, you don’t want that person in QA, because they’re going to miss all those bugs that fall outside of the pre-determined test plans. That type of person should work on an assembly line, or in a sweatshop, instead.
The point of quoting #1, at the top, is also twofold: A) A good engineer is used to debugging, so they should be used to exercising their own code, and B) QA people are not debuggers. Code that’s handed off to them should have a minimal amount of functionality. Their job ideally is to document where the code functions *but* its functionality *isn’t correct.* Crashing bugs *will* inevitably slip through, and it’s their job to find those, too, but the percentage of time they’re finding crashing bugs should be in the low single digits. Crashing bugs cast big shadows that potentially hide other bug creatures that QA people want to jump after and slay, but unfortunately they can’t do their work in the dark.
From a QA person’s perspective, a company that says “we can’t afford a QA department” is a company that expects its *end users* to be their QA department. This is the kind of company that’s not expected to have a reputation for quality products (understatement).
And from a management perspective, I personally don’t want developers taking away time from what they’re hired to do–write code–to do a job that someone else should be hired with QA-specialized skills to do.
I never meant that coders shouldn’t do any testing on their code before handing it off to QA, and I hope my post didn’t come across sounding like I think such a thing. (If it did, then I must have been unclear.)
I think a developer should absolutely do some testing on his/her code before delivering it… but I also know that a developer’s testing doesn’t tend to be as wide-ranging as a QA person’s. A developer is generally thinking, “Does the code I just wrote do what I intended it to, under the conditions I was thinking of when I wrote it?” That’s a far cry from the things real users and real-world scenarios will throw at it.
And throwing that kind of stuff at the code is something I’ve seen lots of QA folks do very well — often in ways that boggle my mind. I get bug reports for all sorts of platforms and situations, and use-cases I would never have come up with. And I’m awed and impressed by the kinds of things I imagine the QA testers thinking, and then doing, in order to provoke such bugs.
QA people are not debuggers, as you point out, and I don’t suggest that they should be. But they’re excellent bug-finders and bug-provokers, which is something that few coders are. (We’re bug-creators as the Cohen quote points out.)
I heartily agree with your last two paragraphs.
To expand a bit, I think I’d say I take issue with the degree of your characterization that “skills, talents, and basic mindset that make a good developer are entirely different from the ones that make a good QA person.”
Being proficient in ferreting out bugs in code is frequently very similar to a QA person discovering that something doesn’t work right and then figuring out how to isolate the bug in order to report it properly.
And a lot of QA–especially anything going beyond basic black box testing–actually involves writing code and scripts, or otherwise “programming” the test harness so that more focus can be put on broadening the overall scope of the testing, and/or focusing on the kind of bug isolation mentioned above that by necessity requires more hands-on time.
Plus there’s lots of QA people who later become developers, for various reasons such as getting away from repetitive black box testing, a desire to make better products when they can fix problems that they find themselves instead of just reporting them, and because developers’ salaries are higher in general.
I think one of the greater distinctions is that there’s a lower barrier for entry to QA, or more generally “testing,” due to basic black box work where a person is expected to just emulate end-users and bang on a product to see if they can make it break. In that regard there’s a world of difference between a “tester”–not so much a “QA Engineer”–and a developer. When you get to the point where people are fully qualified and significantly experienced as QA Engineers, though, I think there’s much less distinction, and then it more becomes different shades of engineering.