Blog Posts Business Management

Unit testing: the hidden part of the test automation iceberg

Blog: Capgemini CTO Blog

Unit testing: the hidden part of the test automation iceberg

These days it is almost impossible to imagine a test automation strategy without Mike Cohn’s test automation pyramid. The pyramid represents optimal test automation distribution, and unit tests should always form its base. Unfortunately, the test automation community focuses on tools and approaches to automate UI tests rather than unit tests. As a result, the pyramid comes to resemble an iceberg, with only the tip consisting of UI tests readily apparent. It is about time to bring the iceberg above surface.

Why focus more on unit testing*?

The test automation pyramid is based on the speed and stability of different tests. Automated UI tests represent the small tip because they are dependent on several factors —  data, browser behavior, connection with a cloud-based platform (like Saucelabs)—which make them slow, brittle, and time-consuming to maintain.

Unit tests verify the smallest pieces of software in isolation by simulating dependencies. This makes them the fastest, most independent and stable automated tests. Unit tests provide immediate feedback on the code, so when something breaks down it is easier to debug. An essential asset in a DevOps environment where rapid feedback is the key. Better still, coding becomes more fun for the developer! It is safer to refactor, and the developer can write code faster while making better development choices.

Why under the surface?

Often I experienced that testing is not seen as part of test automation. It’s part of programming and the responsibility lies with the developers. When application code is written, unit tests are often an afterthought.  I’ve met many developers struggling to come up with relevant test scenarios and write the tests. And actually that is not very surprising, since there quite some pitfalls in unit testing:

– Executing the code without actually testing it.
– Testing the result of simulated dependencies instead of your code.
– Testing third party libraries (like JVM getters/setters, Google API)  instead of your code.
– Missing test scenarios due to complex loops, if/else statements and exception handling.
– Asserting too much in one test which results in difficult debug situations.

Yet testers are hesitant to collaborate on unit testing, but who are they to blame? It is obviously the developers’ job, the tests are hidden in the application code and just way too technical. Besides, if the code coverage is 80%, test coverage is also 80%. Right? Wrong. The only thing 80% code coverage guarantees is that 20% has not been tested and that the code has been executed by unit tests, not actually tested. Code coverage can therefore easily create a false sense of security.

How to break the ice?

Get your unit tests above the surface. Mutation testing is a way to determine the actual quality of your tests. With simple plugins (like PITest for Java or Stryker for JavaScript), little effort is needed to run it against your application code. Your application code is mutated to see if your unit tests fail. Should they pass, the mutation survived and the issue lies with the quality of your test set. In a mutation test report, it is easy to identify the mutation test coverage and to see which mutations survived. This provides actionable input for improvements.

And like always, collaboration is an essential success factor. Collaboration between developers and testers will improve the quality of unit tests. The developer guides the tester through the code and the tester helps the developer creating test scenarios. Meanwhile, the tester learns more about code and the developer learns more about testing: a win-win-win situation, if you’d ask me. As we all know, an iceberg under the surface can be quite dangerous, so collaborate to get it above sea-level!

That being said, I am very curious about your thoughts about unit testing. Would you be interested in learning more about unit testing to get immediate feedback on and improve the quality of your application and collaboration between developer and tester?

*Keynote: API testing; another important part of test automation also lies hidden beneath the surface, but for now I focus on unit testing.





Leave a Comment

Get the BPI Web Feed

Using the HTML code below, you can display this Business Process Incubator page content with the current filter and sorting inside your web site for FREE.

Copy/Paste this code in your website html code:

<iframe src="" frameborder="0" scrolling="auto" width="100%" height="700">

Customizing your BPI Web Feed

You can click on the Get the BPI Web Feed link on any of our page to create the best possible feed for your site. Here are a few tips to customize your BPI Web Feed.

Customizing the Content Filter
On any page, you can add filter criteria using the MORE FILTERS interface:

Customizing the Content Filter

Customizing the Content Sorting
Clicking on the sorting options will also change the way your BPI Web Feed will be ordered on your site:

Get the BPI Web Feed

Some integration examples