Talking vs Communicating

It's easy to conflate talking with communication. Sure, most of the time I would suggest that when people are talking, people are communicating. But the opposite isn't necessarily true. A lack of talking does not necessarily mean there's a lack of communication, or a lack of collaboration. We should be aware of this when making observations about people and the teams they work in.

Phased vs Threaded Testing

There have been many attempts to model different approaches to software testing. You'll be familiar with many of them: exploratory vs scripted; traditional vs agile; testing vs checking; standards-driven vs context-driven; and many more.

There’s another spectrum I’d like to introduce that I’ve derived value from using: Phased testing vs. threaded testing.

Lies, Damn Lies, and Traceability Matrices

Beware the Traceability Matrix. It deceives and misleads.

An Anecdote and a (small) Editorial - The man who opened the door to testing for me

I worry about posting this. I'm worried I'm going to be told to 'check my privilege'. I'm worried I'm going to be told my perspective isn't important. But to add a positive experience isn't to deny that bad experiences happen. I also don't like getting involved in Twitter dramas, but as an opinionated observer, I find their pull irresistible at times. Especially when they involve people I consider to be friends.

There are anecdotes of people leaving the testing field because of James Bach. I'm not denying this has happened. But if we're collecting anecdotes, I'd like to add mine.

An Anecdote

Towards the end of 2009, I had been testing for about two years and I had no public profile. The kind of testing I was doing I would describe now as 'exploratory testing in an agile environment.' I didn't know those words back then though. Describing what I did with the words I knew back I would say "I was doing ad hoc testing in an unstructured environment". I had done ISTQB, gone to local meetups, and interacted with other testers who worked for big corporations. They talked about things like "requirements" and "test scripts" and "traceability matrices". I'd lie in bed at night worrying about what a fraud I was being. You see, I was doing things like "talking to the developers" and "asking them what the most important things to test were." When I was done, I'd report back to them. Not in a test case management tool; I'd walk to their desk and talk to them. Sometimes I'd be as formal as writing an email, or a two-page summary document. I tried testing 'properly' once, but it didn't make much sense to me. I was obviously just not 'getting it' which added to my anxiety.

The STANZ (Software Testing Australia New Zealand) Conference 2009 came around and I begged my employer to go as I was desperate to learn how to 'do testing properly'. One of the speakers was James Bach. Oh he's an expert in the field! His talk was called "Becoming a Software Testing Expert".

Perfect.

In his talk, he talked about "The Tester's Mindset". There was lots of talk about thinking critically, on how to identify threats to value, establishing context. It was immensely useful, and really helped give structure to what I was doing, and let me think critically about my testing and how to improve it.

But it still didn't scratch the itch I had.

So I plucked up the courage to ask him a question after his talk.

"I really enjoyed your talk, and it talked a lot about the kind of testing I'm doing, but, umm.... what if I want to learn how to do testing by the book?"

"Why would you want to do testing by the book?" he boomed back in his American accent. "The book is wrong!"

"I wrote a book about testing, and noone ever checked to see if I knew what I was talking about."

He continued talking to me, right through the break time. The next session was about to start, so he started rummaging through his bag. He said "It looks like we've got to go..." He found what it was he was looking for. "But here, you should read this book. You can have it."

"Exploring Science" by David Klahr. The book James gave me as a faceless attendee at a conference in a remote part of the world

The words "The book is wrong" was the key that opened the software testing door to me. Being gifted a book as a nobody by a gargantuan in the industry was what pushed me through it.

Since then, I've interacted with James countless times:

I've engaged in coaching sessions with him, which he volunteers his time to give (http://www.satisfice.com/blog/archives/393).

The first time I spent one-on-one time in person with him was when he volunteered his time to help Brian Osman set up and be content owner of the first Kiwi Workshop on Software Testing.

The first time I collaborated with James was when he volunteered to co-write an article for Testing Trapeze magazine, and I offered to co-write with him.


These were all 'nice' things for him to do. They came from a place of authenticity.

James can be difficult to interact with, because he doesn't pretend to be nice. I find him intimidating. I have to think carefully about what I say around him. I have to be careful with my ideas. If I say something, I have to defend and justify it. But that's ok. That's who James is, and that's the deal I make when I engage with him. Through this all, I have learnt courage. I have learnt how to be my own biggest critic. Yes, I also suffered from baby tiger syndrome, but I also learnt and grew from that too.

In short, interacting with James is difficult, but worthwhile and I owe much of my career and success so far to James Bach.

That's the anecdote to add to the pile.

A (small) Editorial

I don't think James is a bad person. I don't think James is a malicious person. I do think James is unusual in that he values integrity over manners; honesty over niceness.

I said in the anecdote that I owe much of my career and success so far to James Bach. I think most of the people who would read this have benefited from the work James has done directly, or indirectly, consciously, or not. In fact, I can't imagine what the testing profession would look like if not for his influence on it.

But James is double edged sword, and it frustrates me when I read about people being hurt by his blade, when they don't seem to deserve it. I don't know what to do about that - his sharpness has helped so many; his contribution to the industry immeasurable. But his sharpness has undeniably hurt some people that probably didn't deserve to get hurt. Dulling the blade or sheathing the sword means losing a lot.

Embracing diversity means challenging what is 'usual' and working out how to work with 'unusual'. If we strive to embrace diversity, then we should acknowledge diversity of personality, and diversity of temperaments, and work out how to work with that.

Lean Testing in theory and practice

This article was originally published in an earlier form on the Assurity Consulting website

There are many different definitions of software testing, and many views on what responsible testing looks like in our industry.  How you view the role of a tester informs what practices and artifacts you believe are valuable.

Resources relating to "That's not the map I had in mind"

Resources for "That's not the map I had in mind"

I expect that these diagrams will evolve and grow over time which is why I have included them here for comment. This list will grow over time.

Xmind file for taxonomic hierarchy: XMind file

JPG file for taxonomic hierarchy: Image

 Downloadable examples of various testing models coming soon

WeTest Weekend Workshops 2014 theme: Evolve

This last weekend (29/11/2014), we had our second WeTest Weekend Workshops 2014. The theme was "Evolve".

Shallow KPIs: A Tale of Two Testers

Once upon a time there was a thread on linked in on KPIs for software testers.  A Test Manager shared the KPIs she uses for her team:

1. Amount of bugs created.
2. Amount of bugs verified
3. Amount of assigned work completed.
4. Confirm to schedule. 

(At the risk of accusing anyone on Linkedin of being sloppy with their language, I will assume by 'Amount of bugs created' she means "number of bugs logged in some bug tracking tool").

When challenged, she provided the following 'real life' scenario as if it the sheer power of this example would dazzle us all into submission:

"Tester1 – found 30 defects, verified all assigned issues by deadline.
Tester2- found 0 defects, verified 10% of issues assigned by deadline.
Who performed better Tester1 or Tester2?"

So who performed better?

Tester 2 of course. She didn't log any defects because she had established a strong working relationship with the development team, and as she found an issue, she wrote it on a sticky note, and gave it to the developer. The developer then would rapidly fix and redeploy. The tester would retest, and verify the fix. Because of this, she was able to reduce a lot of administrative overhead, and help the developers produce a high quality product.
Tester 2 was unable to verify all the issues assigned to her by the deadline because she was very thorough, and felt that meeting an arbitrary deadline didn't contribute to the overall health of the project. Instead, she focused on doing great work.

Meanwhile, Tester1 logged many defects. They were poorly written, and many of them were just different symptoms of the same underlying issue. The developers had to spend a lot of time trying to decipher them, and would often spend many hours chasing down bugs that turned out to be merely configuration errors. Once, he logged 10 'defects' that were immediately 'fixed' when someone came over and updated his java environment. A lot of time is spend administering the defects in the bug tracking tool, and trying to work out if Tester 1's defects are legitimate or not.
Tester 1 works very hard to meet the deadline when verifying issues. To do so, he performs a very shallow confirmatory check. His vulnerability to confirmation bias has led him to verify many fixes as "complete" when there were regressive side-effects he didn't pick up on.

Tester 1 meets his KPIs and is up for promotion. In two years he'll be sharing his wisdom in Linkedin.

Tester 2 has been told she isn't performing as necessary. She is going home tonight to update her resume. In a year she'll be working at a company that assesses her performance by watching her work and regularly catching up for peer review. In two years she'll be sharing her wisdom at a peer conference.
 

The Responsibilities of a Conference Facilitator

I have just returned from Let's Test Oz 2014, and like the CAST conferences, operates on a K-Card style facilitation format.

During the three day conference I saw the power a great facilitator can have. I got to experience first-hand the influence a good facilitator can have on the success of a talk, so I would like to offer my perspectives on what makes a good faciliator.

Very Short Blog Post: A date with test cases.

Here's a test case problem:

The requirement:

"Formatting is automatically applied to all date fields (dd/mm/yy formatted)"

 

Here are my findings after a 15 minute test session:

• Formatting is automatically applied when entering dates as

• 12.12.2014
• 12th Dec 2014
• 12 Dec 2014
• 12 December 2014
• 12th December 2014
• 12-12-2014
• 12-DEC-2014

•   Formatting is not applied to:

• 12.12.14
• 12122014
• 12th Dec 14
• 12th December 14
• 12/12/14

a) Did the requirement 'pass'?

b) According to some claims, it is best practice to write one positive test case and one negative test case per requirement. What would I have learned by writing and executing two test cases?

c) Some test management tools would report 100% coverage with 1 test case and if it passed, it would say that the requirement passed.

 

Maybe talking about testing in terms of test cases and of passes and fails isn't useful.

 

 

 

“Anyone Can Be a Tester” - Response

I was pointed to this article on twitter: http://www.morethanfunctional.co.uk/1/post/2014/04/anyone-can-be-a-tester.html by Jari Laakso who has already written a response here.

There are some things I agree with, some things I disagree with, and some things I think are just ugly.

Thoughts around Test estimation

The irony of being asked to provide an estimate as to how long testing is going to take is that, not only is this like asking us how long it's going to take us to find all the easter eggs at easter, but often, it's not really up to us to decide when to stop.

ISTQB: Possum Certification

In my last post, I talked about the concept of possum testing: Doing testing-related activities that the tester does not value, motivated on some level by fear.  I'd like to extend this concept out, and talk about the fundamental problem I have with ISTQB certification: It's a possum certification.

KWST & Possum Testing

(from http://irishchicklette.wordpress.com/2009/02/24/)

This last weekend, I had the pleasure and privilege of attending the first ever KWST (Kiwi Workshop on Software Testing). Seventeen leaders in the software community came together in Wellington to discuss the topic of test professionalism.  You can read about the workshop here and here.  Two very interesting terms came out that Brian Osman had coined to described his observations in the testing profession.  One was stealth testing, which I'll talk about in another post, but the first was "Possum Testing".

"Learn from others' mistakes"

We're always told to learn from others' mistakes.  I read a lot of testing blogs, and take the lessons that other people have encountered and learnt the hard way, and go "phew,  glad I get to learn this the easy way."  But I'm discovering that it's not really good enough, especially when it comes to personal credibility.  I can cite anecdotes from other people and explain why I accept their reasoning and points of view for certain things, but until those stories are mine, and until those lessons are mine, I can't credibly pass those lessons on to others. 

Traditionally, when faced with a situation, I would throw my hands up and go "Nooo, I read about this in Michael Bolton's blog, it's a bad idea cos X, Y, Z!" or "Oh James Bach did a talk on this and why it's bad, let me find it!"  Learn from others' mistakes, right?  Don't repeat them, right?

Well, it may be to the detriment of my own experience.  I'm not seeing first hand the lessons others have learnt the hard way.  So I am now going to consciously try and do things I'm told to do, even if I've read it's bad to do, just to experience it first hand. 

Perhaps I can go into work tomorrow and declare that all our testing should be automated.  Perhaps not. ;)

Inattentional Blindness in action - Anatomy of a testing session

 Inattentional blindness...the phenomenon of not being able to see things in plain sight.  This is how it just happened to me:

Building an exploratory test plan redux: Software cartography

A map is a visual representation of an area—a symbolic depiction highlighting relationships between elements of that space such as objects, regions, and themes. - http://en.wikipedia.org/wiki/Map

Testing software is like exploring new lands.  The same principles apply: There is an unknown something out there, and we want to turn it into a known something.

Putting systems into System Testing

System - A group of interacting, interrelated, or interdependent elements forming a complex whole.  - http://www.thefreedictionary.com/system

People mean different things when they call themselves testers, and people think different things when they think about what it means to test something.  People generally distinguish between unit testing, integration testing, system testing, and user acceptance testing.  But what does system testing actually mean?  Why is it, when we're "taught" about system testing, noone ever explains what a system is?

"Exploratory Testing" is a pleonasm

It's time to stop talking about Exploratory Testing as a subset of testing.  There is no "exploratory testing" and "other testing".  All testing is exploratory.  If it's not exploratory, it's not testing.

Building a test plan from Woe to Go using Heuristics, Mind Maps, and Test Charters

Creating a test plan can be a daunting task, especially when under time pressure.  "We need you to test this application and, oh, we go live next Monday."  How do you create a plan to test that is accountable, transparent, and structured, yet lightweight enough that can quickly be understood by developers, project managers, other stakeholders, and new testers that may be brought on to help out?  And not only that, but be created quickly?