As reported in Donald Firesmith’s Common System and Software Testing Pitfalls, inadequate testing costs “. . . the US economy between $22.2 and $59.5 billion annually . . . “ This accounts for “. . . between 25% and 90% of software development budgets . . .”  That’s a staggering figure that I wouldn’t necessarily want to share with my boss because, well . . . ouch.  

It’s important to note that much of this cost is attributed to software developers and end users in the form of “. . . failure avoidance and mitigation efforts.”  Ostensibly, this is a case for the trained, professional, and methodologically sound QA tester, right?  It certainly seems that way, but money is money, time is time, your boss is dissatisfied with the testing approach, and they’re under pressure to make some changes.  

So what do you do?

Tell them to read this blog, of course!  Before your boss decides to re-structure a department or re-allocate funding, here are 3 things your boss should know about software QA testing.

1. Dynamic Testing is — by itself — not Efficient

I can hear the product owners and executives now: “I told you testing is an inefficient cost center!  Here’s the proof; a QA tester said so in a blog!”  Whoa, horsey . . . 

While it’s true that dynamic testing in and of itself is not an efficient means of assuring system stability, that’s because no single form of testing is.  Dynamic testing (the input-output kind that might center on a UI) is one facet of a much larger, more complex discipline.  However, this is the type of testing we most regularly associate with the concept of “Testing.”  

It’s also (in my opinion) the easiest type of testing to crowdsource with untrained eyes (e.g., users).  This results in some defect identification, but certainly not at the levels necessary to negate the impact of those defects in a reasonable amount of time (i.e., lower cost over time).  Plus, drafting users into the formal QA effort is — honestly — bad form.

2. Dynamic Testing Doesn’t Catch the Most Bugs

Objection, Your Honor!  Dynamically testing the application is the only way to ensure system stability.  After all, if the users can use the application successfully, that means no bugs.

No.

Software QA testing can’t actually prove defects don’t exist.  By extension, it can’t prove system stability.  It can only suggest that, by the negation of defects, that the potential risk of system instability is low.  

So, if dynamic testing can’t make my system stable, what can?

Truthfully, dynamic testing can help to stabilize a system as long as it’s combined with a static testing approach as well.  Remember, software QA testing is a complex discipline.  According to the findings reported by Donald

Firesmith, dynamic testing only accounts for 85% of defects found while static accounts for 95% of defects found.  When combined (like the different elements that make up Captain Planet), great things happen — like 99.27% total testing effectiveness.  

This takes a team of trained testers, though. Implementing a 2-year beta phase with users acting as guinea pigs won’t produce these results.  

3. Test Planning is Crucial

“We don’t need to plan for testing.  If it happens, it happens.  Besides, if the code is good, it won’t need to be tested, anyhow.”

                                      — Not-so-hypothetical product owner

As the old saying goes, if you fail to plan then you plan to fail.  Test planning is crucial because it sets expectations, addresses up-front concerns, and provides a framework that can be used to guide the testing process as well as predict cost.  This makes test planning an indispensable component not only for testing, but for high-level project management as well.

Of the 92 unique testing pitfalls, Fireside cataloged in Common System and Software Testing Pitfalls, 9 of them deal with test planning and documentation.  So, nearly 10% of testing practice dysfunction can be traced back to a lack of planning and documentation.  No wonder Fireside addresses test planning at the front of his book.

So, my questions for you are . . . 

1. What kind of testing is currently taking place on your project?  
2. What kind of testing is not currently taking place on your project?
3. Are your testers trained individuals who know the technique and method of software QA testing?
4. What kind of testing documentation is missing from the current picture?