Wednesday, May 3, 2017

Phased vs Threaded Testing

There have been many attempts to model different approaches to software testing. You'll be familiar with many of them: exploratory vs scripted; traditional vs agile; testing vs checking; standards-driven vs context-driven; and many more.

There’s another spectrum I’d like to introduce that I’ve derived value from using: Phased testing vs. threaded testing.

As the name implies, it describes the testing lifecycle as either a series of phases or as intertwining threads.

I've found this useful in encouraging debate or conversation about testing practice in a way that is agnostic when it comes to SDLC and testing politics, and therefore is a model that can encourage useful introspection without too much baggage.

Phased Testing

Testing has traditionally been modelled and organised as a series of stages that occur more or less sequentially, symptomatic of the waterfall and enterprise contexts in which this tradition was established. When we talk of “traditional” testing, this is usually what is meant, and it often looks like this:

  • Requirements-gathering phase
    • Read requirements documents
    • Collect useful artefacts
    • Establish acceptance criteria
  • Scripting Phase
    • Codify test ideas into artefacts such as ‘Test Cases’
    • Codify test ideas into artefacts such as automated test scripts
    • Codify test “coverage” into a traceability matrix
  • Execution Phase
    • Testers perform testing as described in their test cases
    • Machines execute automated test scripts
  • Exit / Reporting Phase
    • Test exit reports are written
    • Political arguments are had about the coverage and the information found


This sounds fair enough, and the IEEE-829 standard for software test documentation defines a whole document structure to support this model of testing. This phased approach to testing closely mirrors the phase-based waterfall model of software development, but can also be used within agile models of software development as well.

For example, I’ve seen situations where “agile testing” has been defined as little more than writing automated acceptance tests for the acceptance criteria attached to a story; the only difference between this and “traditional” testing being that a machine is executing the test scripts, and not a human. Granted, in agile environments, a tester typically has the opportunity to add value above and beyond the actual ‘testing’ of the product, and the lifecycle may be hours or days, rather than weeks or months, but when it comes to the testing activities per se, you can describe this more accurately as a linear series of phases with little to no iteration. You may argue that’s not ‘really agile testing’, and I’d agree with you, so it’s useful to have a label to describe this pattern independently of the wider SDLC it occurs in.

Therefore, I would like to call this model of software testing the ‘phased’ approach, as it views testing as a series of phases that, for the most part, progresses linearly from one phase to the next. Having to go back to a previous phase is seen as some kind of regression, as some kind of failure to adequately plan, control or execute.

Phased Testing reframed


It strikes me that if we look at the actual intent or purpose of the activities that signify these phases, that we can actually reframe these phases as follows:

  • Requirements-gathering phase  -->  Learning Phase
  • Planning Phase --> Modelling Phase
  • Execution Phase --> Performing Phase
  • Exit / Reporting Phase --> Communication Phase

With this in mind, let’s look at these phases again.

Learning Phase

Why do we gather and read requirements documents, specifications? To learn. This is the phase where we are learning about the project, the problem it is trying to solve, and what has been designed (so far). We collect and read any and all accumulated artefacts, and talk to relevant stakeholders to gain an understanding of what’s going on, and how we are going to test. This phase is not a success simply because we have read documents and accumulated artefacts, it is successful if and only if we have developed sufficient understanding of the project and our contexts to begin productively engaging in testing work.

Modelling Phase

A model is, essentially, something that represents something else in some way. We use a model to help us to understand or manipulate the thing the model represents.

There are many kinds of models. User stories, architecture diagrams, even source code, are all models for the actual software that will be compiled into the machine code that becomes what the user will ultimately interact with.
Test cases are models. They are a model for how a tester or machine will interact with the software and what they will look out for. Therefore, I would like to reframe the “Planning” phase to the more general “modelling” phase, because the success of this phase is not that x number of test cases have been created, but that the tester has developed useful models to facilitate interaction with and evaluation of the product.

Performance

Test Cases are not testing. Testing is not an artifact; it is not something that can be ‘written’.Testing is an event; an activity; a performance. Testing is the evaluation of a product by learning about it through experimentation. Part of the experimentation process and a potential outcome for our learning may be the execution of confirmatory test cases and scripts, but at some point, a performance has taken place.

The success of this phase is not simply that the pre-defined scripts have been fully executed, it is that the product has been interrogated and evaluated to a satisfactory degree.

Communication

Since this is the phase where we produce the ‘test exit reports’ and the pass/fail results, and we receive feedback on these results, let’s call this the reporting/feedback phase.

As a phase, it is often the case that testing occurs in a black hole until this final phase, at which point communication is (reluctantly) engaged in.

This phase though shouldn’t be simply about “exiting” testing, its success should be whether the test group have successfully communicated valuable and useful information about the state of the product to the wider project team and stakeholders.

The apparent absurdity

When we reframe these phases, it starts to appear a little absurd. The idea that we constrain our learning to a phase, and once we’re done learning, we then move onto codifying that learning into test cases doesn’t appear to be a process that fosters continuous learning and innovation. Test cases are opaque. They are hard to understand. Hard to change. Which means that even if we decide to iterate our models after performing some hands-on testing, it is only with great difficulty. And because going ‘backwards’ through the phases is so expensive and painful, we invent structures to help us mitigate change. We have ‘entry criteria’ and ‘exit criteria’ and ‘stage gates’ and ‘sign-offs’.

The phased approach to testing ignores that testing is all about ‘change’.

We change one idea about how something works for another.. We learn more about what people want when we build models, and solicit feedback. We then change our models to incorporate that feedback. Designs change when we learn new things about the problem we want to solve, or the limitations of our proposed solution.

Let’s look to an alternative model of testing. One that embraces and acknowledges change.

Threaded Testing

As an alternative to viewing testing as a series of discrete phases that only move forward (or back only with great pain and difficulty), we can instead consider each of these phases as threads that intertwine and influence one another throughout the entire testing lifecycle.

There is also an element of strategy which, like a cable jacket around wires, guides and directs how those activities occur.

Threads.png



Therefore, we can talk about testing as being ‘phased’ or ‘threaded’ in accordance to the degree that the different threads influence one another, and how iterative the approach is.

In this model, change is embraced, and we let new information influence our behaviour. We let our models adapt and evolve, and we perform testing to learn, and to get feedback on what we’ve learnt. Of course, with a model of testing so malleable, we need structures and techniques that lend themselves to rapid adaptation, which is why you’ll find me often advocating for visual test management tools such as mindmapping software, and approaches to test performance closer to the exploratory end of the spectrum.



In the past, I’ve talked about ‘lean testing’, and I now consider a threaded testing approach to be a key tactic in achieving a lean testing approach, almost to the point of being synonymous.

The intention to describe testing as either phased or threaded is to be agnostic when it comes to SDLC and testing politics, and to encourage useful discussion without too much of the baggage sometimes associated with other terms. I have used it when I’ve wanted to talk about changing testing paradigms without getting derailed or sidetracked by other terms which can be politically loaded in the environments I sometimes operate in.

Thanks to Adam Howard (@adammhoward) for his patience in reviewing the writing and rewriting of this post

52 comments:

  1. Thanks for resolving the knot in my brain by putting these thought down. Very clearly expressed.

    ReplyDelete
  2. Selenium WebDriver fits in the same role as RC did, and has incorporated the original 1.x bindings. It refers to both the language bindings and the implementations of the individual browser controlling code. This is commonly referred to as just "WebDriver" or sometimes as Selenium 2.
    Selenium Training Institute in Chennai | Best Selenium Training Center in chennai

    ReplyDelete
  3. Things are very open and intensely clear explanation of issues. Operating System Project Help Service was truly information. Your website is very beneficial.

    ReplyDelete
  4. Good way of telling, good post to take facts regarding my presentation subject matter, PHP Programming Homework Help which i am going to deliver in my college

    ReplyDelete
  5. you can empower them to twist up recognizably better than anything they to think they are, Proessay.co.uk "We have to welcome that managing the results of direct is the best way to deal with change it. I'm upsetting at posting routinely; I don't justify the blog accomplishment! in any case, I saw your blog, its genuinely surprising, I like it, If you have faith in others and give them a positive notoriety to maintain, you can enable them to wind up noticeably superior to anything they to think they are.

    ReplyDelete
  6. Great blog.you put Good stuff.All the topics were explained briefly.so quickly understand for me.I am waiting for your next fantastic blog.Thanks for sharing.Any coures related details learn...

    software testing training in chennai

    ReplyDelete
  7. Enormous blog. You put high-quality material. every one the theme were give details for a short time. so rapidly appreciate for me. I am to come for your after that improbable blog. recognition for allotment.yellow onyx

    ReplyDelete
  8. After reading this web site I am very satisfied simply because this site is providing comprehensive knowledge for you to audience.
    Thank you to the perform as well as discuss anything incredibly important in my opinion. We loose time waiting for your next article writing in addition to I beg one to get back to pay a visit to our website in


    Selenium training in Chennai
    Selenium training in Bangalore
    Selenium training in Pune
    Selenium Online training

    ReplyDelete
  9. https://tnhboxes.com/
    https://www.paperslibrary.com/
    https://topacademictutors.com/
    https://liveassignmenthelp.com/
    http://rankhawks.com/
    http://www.quotehawks.com/
    https://cosmocarparts.com/
    http://aiwah.pk/

    ReplyDelete
  10. Thank you for taking the time to write about this much needed subject.Thanks for sharing.

    custom Packaging Boxes usa

    ReplyDelete
  11. i really impress your work. Thanks a ton for sharing this information in this article.
    Custom Boxes

    ReplyDelete
  12. Vanskeligheter( van bi ) vil passere. På samme måte som( van điện từ ) regnet utenfor( van giảm áp ) vinduet, hvor nostalgisk( van xả khí ) er det som til slutt( van cửa ) vil fjerne( van công nghiệp ) himmelen.

    ReplyDelete
  13. This comment has been removed by the author.

    ReplyDelete
  14. Thanks for one marvelous posting! I enjoyed reading it; you are a great author. I will make sure to bookmark your blog and may come back someday. I want to encourage that you continue your great posts.web designing training in bangalore

    ReplyDelete
  15. Very interesting, good job and thanks for sharing such a good blog. your article is so convincing that I never stop myself to say something about it. You’re doing a great job. Keep it up…

    Softgen Infotech is the Best Oracle Training institute located in BTM Layout, Bangalore providing quality training with Realtime Trainers and 100% Job Assistance.

    ReplyDelete
  16. Wow What A Nice And Great Article, Thank You So Much for Giving Us Such a Nice & Helpful Information about Java, keep sending us such informative articles I visit your website on a regular basis.Please refer below if you are looking for best Training Center.
    Java training in chennai | Java training in annanagar | Java training in omr | Java training in porur | Java training in tambaram | Java training in velachery

    ReplyDelete
  17. The Pros and Cons of Group Health Insurance By Timothy Hebert | Submitted On May 09, 2010 Suggest Article Comments Print ArticleShare this article on FacebookShare this article on TwitterShare.# BOOST Your GOOGLE RANKING.It’s Your Time To Be On #1st Page
    Our Motive is not just to create links but to get them indexed as will
    Increase Domain Authority (DA).We’re on a mission to increase DA PA of your domain
    High Quality Backlink Building Service
    1000 Backlink at cheapest . With that in mind, this article will investigate the upsides and downsides

    ReplyDelete
  18. I think this is the best article today. Thanks for taking your own time to discuss this topic, I feel happy about that curiosity has increased to learn more about this topic. Keep sharing your information regularly for my future reference.Excellent blog admin. This is what I have looked. Check out the following links for Software testing companies USA
    Test automation software
    Best automated testing software
    Mobile app testing services
    Load testing services

    ReplyDelete
  19. Leads generated before the event accounting for about 80 percent of leads at 2-day events and about 60 percent at 3-day events which This makes sense and given that there are no sessions or other meetings competing for attendees' attention until the event starts. virtual edge and thank you letter for attending event

    ReplyDelete
  20. Thanks for sharing this type of information. I am hunting for this information and find the key aspect of this work effective and trustworthy for a valuable answer. Being a USA scholar, I am always in the search of Assignment Helper in USA USA for a better answer. So, you do not lose passion and do the mandatory change in its work. our service is available to you at the most reasonable. Do not stress anymore and take our help.

    ReplyDelete

  21. This amazing blog post! This is very interesting; you’re a very skilled blogger. I’ve joined your feed and look forward to seeking more of your excellent post. A million thanks for sharing. lasuth postgraduate in echocardiography admission form out

    ReplyDelete
  22. Hope walk record. Feeling find help something family. Theory standard hold west key little entire above.education

    ReplyDelete
  23. Official specific after rise run can page. Edge which space wide feeling.religious

    ReplyDelete