Before we begin

"Many arguments appear to be about conclusions, when they're almost always about premises."

Before I begin, I think it's important to lay out exactly what I mean when I use certain words and phrases. Then if you disagree with me, you can decide if we're arguing about conclusions or premises. :)



Testing - I use Cem Kaner's definition of testing:
Testing is an empirical investigation conducted to provide stakeholders with information about the quality of the product or service under test.

To me, testing is more of a social science than a computer science. So what does that definition above actually mean? Let's break it down:

Empirical Investigation: "Based on, concerned with, or verifiable by observation or experience" (Oxford English Dictionary)

Let's look at Adriaan de Groot's empirical cycle:

from Wikipedia: http://en.wikipedia.org/wiki/File:Empirical_Cycle.svg

 
Observation: Observe some attribute of the application: "I observe a screen with two text input fields.  One has "username" next to it, and the other has "password" next to it. There is a "login" button and there is a "register" hyperlink.
Induction: Formulating hypothesis: In my experience, screens that have these fields in them are called "login screens" and allow registered users to access content and restrict non-registered from accessing the content.
Deduction: Creating a testable prediction based on your hypothesis:  I have not registered for this site, therefore, if I enter my login details, I should be denied access.
Testing: Test the hypothesis:  I enter "aaronhodder" in the username field, and "password" in the password field.  "Access Denied" displays.
Evaluation: Evaluate the outcome of the testing: It behaved how I expected it to behave. I have evidence that this screen may be working in a way that is consistent with other web applications.

But we're not finished

Observation: "Access denied" displays
Induction: "Access denied" displays when any user, registered or not, enters their login details
Deduction: If I enter the details of a registered user, then "Access denied" will display
Testing: Obtain the details of a known registered user and enter them in the text fields.  I am allowed access to the restricted part of the site
Evaluation: Hypothesis was incorrect, and I have further evidence that the screen may be working in a way that is consistent with other web applications
etc
etc
and so the cycle continues.

Note that the definition I use puts the tester's skills at evaluation and judgement at the centre of the testing process, not tools, or scripts.  Context over process.  Requirements over requirements documents.

Stakeholders: Developers, project managers, customers, clients, the business.  Anyone who holds a stake in the project.  It is important to know who your stakeholders are, and what is important to them to ensure that the information you are providing is useful.
Information: Examples of information: Does it do what we claim it does? Does it do what the customer wants it to do?  What defects have we found?
Quality: Here I use Gerald Weinberg's definition: Quality is value to some person with Cem Kaner's addendum: ...that matters.  Quality is value to some person that matters.  Part of the tester's role is to discover who the people are that matter, and what they value.

So that's the sum of its parts. The whole is of course much larger, and that is that testing is an active investigative process to discover and report important information about the product or service to the people that ought to know it.


I hope this makes future posts easier to understand and debate by framing it in the above context.