Building an exploratory test plan redux: Software cartography

A map is a visual representation of an area—a symbolic depiction highlighting relationships between elements of that space such as objects, regions, and themes. - http://en.wikipedia.org/wiki/Map

Testing software is like exploring new lands.  The same principles apply: There is an unknown something out there, and we want to turn it into a known something.

When exploring new lands, there are certain properties we are interested in.  We may be interested in physical features such as coastlines, mountains, and rivers, or we may be interested in political borders, we may be interested in underwater features, we may be interested in rock distribution and faultlines, or we may be interested in all these things.  And how do we report on what we find, and what our understanding of the land is, based on the properties we're interested in?  We map them.

1. Identify Properties you're interested in
James Bach's Heuristic Test Strategy model is a great resource for stimulating test ideas.  Identify the properties we're interested in, or may be interested in (after all, in these early days, who knows what may be important.  We may not set out being interested in rock formations, but that probably would change should we find a large coal seam) and begin the framework for your map:

2. Start collecting intel
Before setting sail for uncharted lands, we should gather as much information as we can, after all, we are in a state of ignorance about what we will encounter.  There may be tales of great mountains, of bountiful oceans, and of savage beasts.  There may be scrolls written by soothsayers, and there will be accounts written by previous travelers.  We should sketch out our map based on this so we have a frame of reference for our journey, but fully expecting that we will change it, add details, take away wrong details etc when we get in there.  A scripted traveler would take this heresay and folklore, and immediately chisel their map in stone, and describe in detail their path through these unknown lands.

When we begin a testing project, we are in a state of ignorance.  We should gather as much information as possible.  There may be specification documents,  Responses to RFPs, wiki pages, High level descriptions, requirements documents.  There may be developers who can tell us great tales, and BAs with promises of bounty.  We should begin mapping now so we have a frame of reference for our journey, but fully expecting that we will change it, add details, take away wrong details etc when we get in there.  A scripted tester would take this heresay and folklore, and immediately chisel their map in stone, and describe in detail their path through these unknown lands.

Map taking form as more information is discovered.

The act of mapping sparks new questions allowing you ask more focussed questions of the people around you.  The map becomes the tool to make the map better.  This is the beauty of an organic system.  Give it some initial conditions (the heuristic test strategy model) and it almost builds itself.  Here is the same map after a conversation with a developer:

New parts have been added, We've learnt a bit about the structure, and about the platforms, and what is required to install it all.  We were able to ask good questions because we could see where the silhouettes were that needed to be illuminated.

3. Begin touring
We feel we have a useful framework to begin the actual work of touring the product.  We know where the general inlets are, where the major landmarks are; we have enough to make landfall.  Now we get the lay of the land.  "Oh here's a river we didn't know about, better map that so we can take a closer look later." "This mountain is a surprise, I wonder if anyone knows about it".
As an example, as I installed something, it asked me for a port number [3306].  Interesting, I better map that as a test idea:

Eventually your map will be fleshed out that you're pretty confident that it is a pretty good and useful model of the software under test.

4. Create Test Charters

If you're unfamiliar with test charters, or session-based test management, read these:

http://www.satisfice.com/sbtm/

http://www.satisfice.com/articles/sbtm.pdf

http://www.satisfice.com/articles/et-article.pdf

With a little tweaking, you've almost automatically generated your test charters from your mind map.  As you run along the lowest levels of hierarchy, create your test charters from them.

For example, one of my test charters will be "Install the Schedule Database with ports other than 3306".

5. Keep adjusting the map
Your map is a model.  As your model changes, so will your map.  Parts that you thought were important will be deleted, and parts you didn't know existed will rise to prominence.    The map is a reflection of YOUR understanding of the product at a given time.

It also has other uses.  You can use it to manage the testing process by assigning testers to a branch, letting them take ownership of all sub branches.  This lets you quickly assign work without having to go one by one through requirements and specifications and manually assigning individual tasks.  You could colour branches that you're currently working on, and shade in branches that have been completed.  At a glance, you can see what has been done, what is being worked on, and what is left to do.  

I hope this illustrates a good way to structure, manage, and report on exploratory testing.  The main advantage is that it is quick to do:  It is extremely lightweight and can be easily added to, and parts moved around with ease.  It also doesn't need to be "read".  A multipage document requires sitting down to read, and it is left to the reader to try and interpret what you mean.  A map can be glanced at and almost immediately the structure, connections, functions, what needs testing, what hasn't been thought of are apparent.  A map is worth a thousand words.


The software I used in this post is the free version of XMind, but there are lots of different tools you can use.  Freemind is also a good one.