Thursday, February 17, 2011

Building a test plan from Woe to Go using Heuristics, Mind Maps, and Test Charters

Creating a test plan can be a daunting task, especially when under time pressure.  "We need you to test this application and, oh, we go live next Monday."  How do you create a plan to test that is accountable, transparent, and structured, yet lightweight enough that can quickly be understood by developers, project managers, other stakeholders, and new testers that may be brought on to help out?  And not only that, but be created quickly?



I'd like to propose a solution by combining knowledge and techniques from James Bach and others in the context driven community to create a solution which may help, depending on the application under test, and your workplace culture.  It sometimes works for me, and I hope that it may work someone else.

1. James Bach's Heuristic Test Strategy Model
Available here: http://www.satisfice.com/tools/satisfice-tsm-4p.pdf
As it says on the front page: "The immediate purpose of this model is to remind testers of what to think about when they are creating tests."


I like to build from the Quality Criteria categories, although I think you could begin building from any of the categories, depending on the type of model you want to build.

2. Mind Mapping
There's a lot of talk in the context driven testing community right now about mind mapping, and I have to admit I was originally dismissive of the idea.  I had always associated mind mapping with high school social studies teachers for some reason, or, at best, highly paid advertising executives, and wondered if they really had a practical purpose in the 'real world'.  I've since learned, of course, there is no such as the 'real world' so I started looking into it.  There are a few free mind mapping tools out there: http://en.wikipedia.org/wiki/List_of_mind_mapping_software
In these examples I'll be using FreeMind
You can read more about using mind mapping in testing here: http://www.bettertesting.co.uk/content/?p=956

3. Combine
Select the appropriate criteria for your project and map it out (I use the word 'Functionality' instead of 'Capability'):








                     4. Flesh Out
Start fleshing out using the other categories to generate ideas.


5. Post up in a public area
When you think you've finished, or you've run out of ideas, print out, and post up somewhere visible.  This lets PMs and developers see what your model of the application is, and they can point out areas that you may have missed.

6. Create Test Charters
If you're unfamiliar with test charters, or session-based test management, read these:
http://www.satisfice.com/sbtm/
http://www.satisfice.com/articles/sbtm.pdf
http://www.satisfice.com/articles/et-article.pdf

With a little tweaking, you've almost automatically generated your test charters from your mind map.  As you run along the lowest levels of hierarchy, create your test charters from them.  I find that usually each lowest level generates 1 to 3 test charters.  In the example above, some example test charters may be:
  • Create admin user; ensure has admin rights as described in document X, and can create, edit, and delete users
  • Check the claims made by the sales team are present in the application
  • Test the Proin at ligula libero
  • Explore the Quisque quis libero urna using Internet explorer 6
7. Managing the testing process
The mind map lets you visualise the testing process and can be used to report how testing is going.  You can use it to manage the testing process by assigning testers to a branch, letting them take ownership of all sub branches.  This lets you quickly assign work without having to go one by one through requirements and specifications and manually assigning individual tasks.  You could colour branches that you're currently working on, and shade in branches that have been completed.  At a glance, you can see what has been done, what is being worked on, and what is left to do.  

I hope this illustrates a good way to structure, manage, and report on exploratory testing.  The main advantages are that it is quick to do: no lengthy documents with headings and paragraphs of text; and has many emergent properties which also aid the management of the testing process.

10 comments:

  1. Interesting thoughts and a real challenge to my approach to knocking up a test plan in short order (take an old one and do a search and replace on the project name!)

    What is your experience at the 'Make It Public' stage. I have difficulty getting the right people to review and approve a standard plan (however short), Do you find that your people understand the mind map and respond or do you get a low response?

    ReplyDelete
  2. I agree posting the mindmap where it is visible can help focus on charter much better. Just like Agile developers use Scrum and Kanban boards to check the progress within a sprint. The only downside I see is that map becomes bigger and bigger as more and activities are added and to some extent unmanageable. If you have a large posted area then it helps but most workspaces are so cramped that you don't find the room to display a large map.

    ReplyDelete
  3. Hi Aaron,

    Some good ideas here, I really like the collaboration view point of sticking the mind map in a break out area for feedback. That's very good, and cheap.

    I'm not keen on using the mind maps to track planning aspects though; I think that could become confusing.

    Lean reporting is something I'm very interested in, and something I'll be working on for our team over the next six months. So it's really helpful seeing ideas like this.

    I think the real challenge is getting something that’s not only lean, but actually fits everyone's needs. It's often a struggle getting a format together that fits all the recipient’s needs.

    Thanks for taking the time out to share your idea with us.

    Cheers,

    Darren.

    ReplyDelete
  4. @roqueconsulting: Yeah this is very workplace culture dependent. I guess it depends if you need things approved to proceed. Either way, in whatever format you need to eventually present, this could still be a tool to generate good plans and to use for your own planning. You could have your "official" document to tick the boxes, and your "useful" document to actually get your work done.

    ReplyDelete
  5. @Mohinder: Yeah, the size and manageability can become unwieldy. I'll let you know how I get on as the map grows

    ReplyDelete
  6. @Darren McMillan: I'd be really keen to hear more ideas about how to make this work. This is something I'm only just now beginning to explore.

    ReplyDelete
  7. I've followed the same approach in a startup environment and what I also found useful were checklists that kinda followed the mindmap to do regression testing

    Thanks for the read

    ReplyDelete
  8. Hi Aaron,

    Excellent post. You suggest this is a good way of doing planning when time is short and deadlines loom. Do you think it would be a good strategy for a longer term, larger project (like a traditional waterfall) one?

    I've used them a lot for bigger projects too with mixed success (mainly because of management/stakeholder expectations).

    Nice post.

    Thanks
    Rob..

    ReplyDelete
  9. @Rob Lambert: I can't see why you wouldn't use this for a longer term project. It means you never lose sight of the bigger picture as the weeks and months go by. It can also have quite a visual impact as new requirements and features are added, and the map grows like a weed. Really shows the cost of new features in terms of testing time.

    ReplyDelete
  10. I'm so glad to have found such a clear and detailed publishing. Building a test plan is a daunting task, but your article can be helpful in this case. Follow this link in order to see what we can do for your academic success!

    ReplyDelete