You need to differentiate between ‘testing’ and ‘checking’. A check is when you verify something to confirm existing beliefs. You check when you want to verify something still works the way it did before. You aren’t learning anything new, you’re verifying old knowledge.
‘Testing’ is exploratory. You ‘test’ when you want to find out new information. Testing is investigating the application to discover information the help people make quality and risk-based decisions about the application under test.
Checks can be automated. Tests cannot. Tests require a skilled tester to engage their brain and question and evaluate the application. Checks can be done by a machine.
Why the fetish with automation? Why would you want to rerun an old test when you can execute a brand new test? Why would you want to keep feeding the same data in, when you can be trying to invent new ways to test your system? Why would you spend weeks writing scripts that could be designed, and executed by a human in minutes (or seconds?).
Yes, in terms of regression, good checks are important to have, and are good to automate, but why would your goal to try and be to restrict your testers from finding out new and interesting things, and only getting a machine to tread the same paths again and again?
Yes, in terms of regression, good checks are important to have, and are good to automate, but why would your goal to try and be to restrict your testers from finding out new and interesting things, and only getting a machine to tread the same paths again and again?
Not only that, but checks can only find things that are specified. But a tester can find potential problems no one has even thought to look for just by the simultaneous learning, design and execution of tests. Serendipity is a powerful ally.
Would you rather write about a test, or actually perform a test and learn something about the application which may lead you to design and perform more interesting tests?
The goal of automating as many tests as possible is mistake, and if this is how you feel you really need to question why you feel that way.
The goal of automating as many tests as possible is mistake, and if this is how you feel you really need to question why you feel that way.
Hi Aaron!
ReplyDeleteI disagree that doing checking testers are not learning new things. In order to write checking code in some scripting language (or tool) you need to know how to do this. You are learning about system under test by reading requirements, wiki pages or by talking with developers.
Doing the exploratory testing you are thinking about putting system under test in action that is product of your imagination but also based on knowledge that you had collected during the development of checking scripts.
Also exploratory test can be automated, but only after you designed it (using your knowledge, imagination, ...).
Regards, Karlo.
"You are learning about system under test by reading requirements, wiki pages or by talking with developers."
ReplyDeleteIf you are reading about them in requirements, wiki pages, or by talking to developers, then it is not new information. It is information already known.
"Doing the exploratory testing you are thinking about putting system under test in action that is product of your imagination but also based on knowledge that you had collected during the development of checking scripts."
I'd say it's the other way around. People develop scripts based on the knowledge they learnt during exploratory testing.
"Also exploratory test can be automated, but only after you designed it (using your knowledge, imagination,"
The execution can be. I can't really comment, but Michael Bolton wrote something that supports the statement "Exploratory testing can be automated". I think you'll find it depends on how you define the question though: http://www.developsense.com/blog/2010/09/can-exploratory-testing-be-automated/
checking vs testing is one perspective
ReplyDeletemanual vs automated is another
additional perspectives can be found in Crispins Vancouver quadrant.
Full automation is about as possible as finding all bugs. There is a long tail of bugs and a long tail of testcases.
http://www.eurostarconferences.com/blog-posts/2010/9/6/close-the-tool-and-start-exploring---jesper-ottosen.aspx
"If you are reading about them in requirements, wiki pages, or by talking to developers, then it is not new information. It is information already known."
ReplyDeleteYou are right about that this in not new information. But my point was that this is starting point. You need to gain that knowledge in order to revel new information using exploratory testing.
Jesper is right - I think you'd find the Testing Quadrants used in Agile really educational. It kind of highlights areas where automation works best and where it does not. And the best kinds of testing use a bit of both.
ReplyDeleteI like you're article. You're kind of right, the kind of tests you do with automation are just a kind test you probably expect to pass. But automation removes the donkey work from testing - when you're running a test for the 6th time you tend to go "yeah yeah yeah I know this" a lot.
That said, there's always a place for human intuitive "playing around". However the best place for that is after you've run the automated tests.
Picture of those quandrants ...
ReplyDeletehttp://onestepbacktwostepsforward.blogspot.com/2009/06/agile-testing-quadrants.html
Allassignmenthelp is a web portal where students get assignment help for all the subjects, with the help of our experts. You will get 100% plagiarism free assignment. Expert’s consultation is also available for students. If they have any query they can contact with our experts anytime.
ReplyDeleteassignment help online
Our iOS Application Development Company keeping in respect the particular plan and advancement rules identified with every single individual stage - iPad, iPhone, Apple TV, and Apple Watch. We have constructed applications for assorted use cases going from continuous examination to course coordinating, geo-labeling, video web based, and that's just the beginning.
ReplyDeleteYour such a wonderful web-blogger. Keep it up as I'm expecting some more from this site. Thank you so much for sharing. Also visit adsu school fees schedule for part-time students
ReplyDelete