Sunday, April 6, 2014

“Anyone Can Be a Tester” - Response

I was pointed to this article on twitter: by Jari Laakso who has already written a response here.

There are some things I agree with, some things I disagree with, and some things I think are just ugly.

Stuff I agree with

"I recommend you get as many non-testers to test your software as possible."
I completely agree with the sentiment here.  Every person who interacts with your product brings their own history, their own quality criteria, and their own biases. As trained professional testers we try to be conscious of our biases but they cannot be eliminated completely. And over time, we become familiar with the interfaces, consciously or unconsciously work around quirky behaviour, and become blind to the familiar. Getting different perspectives can be a very Good Thing, and will augment your testing effort.
Rarely replace.
And bear in mind that somebody still needs to design the testing sessions, collate that information, and present it in a meaningful way.

"You can use as many testing techniques and tools as you want but if it isn't very usable, the software's crap."
Again, I agree principle. Bearing in mind that what is useful for one user may be completely unusable for another.  So, assuming you mean "...if it isn't very usable for the people who will be using it, the software's crap."  Collecting usability and user experience data from an audience of "developers, product managers, directors and whoever else we could find" may be of limited use if the target user group are power plant engineers, the elderly, or toddlers, for example.

"I was shocked at how usability and user experience weren't mentioned once."
I guess your 'thing' can be to bring usability and user experience into common testing discourse, then. That's a positive opportunity.

Stuff I disagree with:

 "anyone can test software"
Whenever someone scratches their head and tries to work out how to indent their bullet list in MS Word without indenting the paragraphs around it, they are testing. Whenever someone is trying to work out what a menu option does, they are testing. When someone downloads an app and is assessing whether it meets their needs and desires, they are testing. So yes, to that extent, anyone can test software.


...I don't think that's what you meant. While anyone can interact with software and make some meaningful observations about it, a professional tester does it methodically, precisely, and within the overall mission of testing. That means making judgements about what to test and how to test it based based on knowledge of the stakeholders and what they value, and a whole bunch of other context-related criteria while being mindful about responsible use of time and money.

Yes, anyone can wander around a product and bump into the odd bug, some of which may be valuable. But not anyone can actively investigate a product in a methodical way and be able to justify what they will test and how, and what they will not test, to discover threats to value and present it in a meaningful way to stakeholders, while working within project constraints.

Most people probably have the capacity to learn testing in this way, but not everybody has learnt testing in this way.

The ugly

"when I've heard them say it, there are normally other testers nodding their head or concurring in a "yeah, everyone who isn't a tester is a dick" kind of way."
OK, that's your projection, not mine.
 "After filtering out non-issues and duplicates (one developer logged a bug that had been assigned to him for over a month) there were over 100 bugs logged.
Comfortingly for me and my boss, there were no massive issues logged."
I consider a professional software tester to be able to critically analyse those statements, and be able to demonstrate how the statements are meaningless out of context, and may even be used against your argument. Not everyone can do that.

I kindly suggest that if you don't want to alienate others, and you want people to take your message seriously, perhaps you should rethink statements such as these:
"you might as well take your test sessions, your mindmaps and your automation scripts then insert them promptly and firmly up your arse."
This makes you sound like a dick.  Maybe you want to sound like a dick, I don't know, but in case you don't, I'm letting you know.
 "...and using pretentious terms like 'test charter' (that means a test objective or mission for anyone who doesn't attend the church of James Bach)."
"However, software testing recently seems to be all about using the fashionable terms and techniques instead of doing what a tester is supposed to do"
I'm sorry you haven't heard of a term that's been around for over thirteen years. But that doesn't make it pretentious. But you're not alone in not hearing about these 'fashionable' terms. Which is exactly why the community appears to harp on about them. Because those of us who use them have discovered they work really well, and we want to share what we've learnt.

In Conclusion

I like your message that "we should be asking as many people as possible to test the software we're working on and listening to their feedback." I agree that there can be too much emphasis on the 'functional' aspect of software testing. But that message is completely obliterated by all the other stuff in there.

1 comment:

  1. Hi Aaron, thank you so much for reading my blog and taking the time to comment on it.

    I'd like to respond to the bad and the ugly points you make.

    Your first interpretation of what I meant by 'anyone can test software' was actually right on the money. I'm genuinely not saying than anyone can be a good, professional software tester. I mean that anyone can use/test the software and provide valuable information from doing so.

    The comment about peoples thoughts on non-testers is the attitude I have seen in a fair few testers. It isn't a projection, it's just my interpretation of the tone and way in which some testers talk about non-testers sometimes.

    You're right in that the methodology-up-arse comment was supposed to be somewhat provocative. That was more a result of writing when in a bad mood. I've taken your comments on board and will endeavor to show more maturity in future entries.

    My issue isn't that I don't understand what a test charter is, I've used them when we tried SBT in my current job last year. The problem I have with it is that if I say 'test charter' to a developer or PM or whoever, they don't know what I'm talking about. If, like the examples I gave, 'test mission' or 'test objective' was used instead, it would less likely isolate people not savvy to SBT.

    If I've missed anything, I apologise.

    Thanks again for posting this response to my blog entry, it has both provided some valuable feedback as well as put the theme from The Good, The Bad and The Ugly in my head. Both are good things.