Thursday, December 16, 2010

Before we begin

"Many arguments appear to be about conclusions, when they're almost always about premises."

Before I begin, I think it's important to lay out exactly what I mean when I use certain words and phrases. Then if you disagree with me, you can decide if we're arguing about conclusions or premises. :)

Testing - I use Cem Kaner's definition of testing:
Testing is an empirical investigation conducted to provide stakeholders with information about the quality of the product or service under test.

To me, testing is more of a social science than a computer science. So what does that definition above actually mean? Let's break it down:

Empirical Investigation: "Based on, concerned with, or verifiable by observation or experience" (Oxford English Dictionary)

Let's look at Adriaan de Groot's empirical cycle:
from Wikipedia:
Observation: Observe some attribute of the application: "I observe a screen with two text input fields.  One has "username" next to it, and the other has "password" next to it. There is a "login" button and there is a "register" hyperlink.
Induction: Formulating hypothesis: In my experience, screens that have these fields in them are called "login screens" and allow registered users to access content and restrict non-registered from accessing the content.
Deduction: Creating a testable prediction based on your hypothesis:  I have not registered for this site, therefore, if I enter my login details, I should be denied access.
Testing: Test the hypothesis:  I enter "aaronhodder" in the username field, and "password" in the password field.  "Access Denied" displays.
Evaluation: Evaluate the outcome of the testing: It behaved how I expected it to behave. I have evidence that this screen may be working in a way that is consistent with other web applications.

But we're not finished

Observation: "Access denied" displays
Induction: "Access denied" displays when any user, registered or not, enters their login details
Deduction: If I enter the details of a registered user, then "Access denied" will display
Testing: Obtain the details of a known registered user and enter them in the text fields.  I am allowed access to the restricted part of the site
Evaluation: Hypothesis was incorrect, and I have further evidence that the screen may be working in a way that is consistent with other web applications
and so the cycle continues.

Note that the definition I use puts the tester's skills at evaluation and judgement at the centre of the testing process, not tools, or scripts.  Context over process.  Requirements over requirements documents.

Stakeholders: Developers, project managers, customers, clients, the business.  Anyone who holds a stake in the project.  It is important to know who your stakeholders are, and what is important to them to ensure that the information you are providing is useful.
Information: Examples of information: Does it do what we claim it does? Does it do what the customer wants it to do?  What defects have we found?
Quality: Here I use Gerald Weinberg's definition: Quality is value to some person with Cem Kaner's addendum: ...that matters.  Quality is value to some person that matters.  Part of the tester's role is to discover who the people are that matter, and what they value.

So that's the sum of its parts. The whole is of course much larger, and that is that testing is an active investigative process to discover and report important information about the product or service to the people that ought to know it.

I hope this makes future posts easier to understand and debate by framing it in the above context.


  1. Good post and good luck for the many many post to come.

  2. Awesome post Aaron - welcome to the blogsphere and looking forward to more posts especially after such an excellent start!

  3. I'm puzzled with lots of exercises. I was afraid I could not do the right time despite my hard work. I need a support person.

  4. Assignment Help online
    Try to add only relevant data and information. If you feel that you are stuck, you can take assignment help online to get a proper idea. There are several web services which can help you to find out a suitable solution to your problems.

  5. you can go through Reviews. It will help you to know about the services, and it will help you to know whether that site is reliable or not. This is a way of understanding the opinion of users, and it helps the website to maintain its functionality and reliability.


  6. I found this one pretty fascinating and it should go into my collection. Very good work! I am Impressed. We appreciate that please keep going to write more content. We are the assignment helper, we provide services all over the globe. We are best in these :- services
    Essay writing
    Essay Writer
    Article rewriter
    Essay writing service
    Essay writing help
    Write My Essay
    Write My Article
    hire writer
    Write my essay cheap
    hire article writer

  7. Thanks for sharing this information. I have shared this link with others keep posting such information. to provide best in class law assignment help online at very affordable prices.

    Assignment Help
    Assignment Helper
    Essay writing
    Essay writing service
    Dissertation help
    Thesis writing help
    Write My Essay
    Computer Science Assignment Help
    Assignment Help South Africa
    Assignment Writing Service

  8. Many of the people are depressed about the problems of essay writing. Well, don’t worry about that because we are providing this service at a very reasonable price.
    Digital Markrting Blog
    SEO Blog
    Social Media Blog
    Email Marketeting Blog
    Online marketing Blog

  9. I loved the article, keep updating interesting articles. I will be a regular reader I am offering assignment help to students over the globe at a low price.
    Do My Homework
    Do My Assignment