Do you really love testing?

We've been having a high-level discussion on testing as a whole at SpikeSource (no big surprise there really) of late. A number of points of views have been flying around, prompting Nagu to write the "Why I love testing" entry on the SpikeSource Developer's site.

From a developer's point of view, I argued that no-one really likes testing. It's the dull end of development and no matter how you dress it up, it is never going to be sexy. It's the housekeeping of coding; the hoovering or washing-up if you like. It has to be done, but more often than not, it is left to the last minute and only done out of necessity.

The code-coverage figures we produce every night on the open source components we test in our framework bears this out. Very few components make it past the 50% mark in terms of their code being exercised with their tests.

Now reading Nagu's post, you find yourself nodding with everything he says. But one thing to note, he merely talks of the benefits of testing, not the actual production of the tests. Sure we all love the outcome that testing produces; greater stability and reliability. Let's put it this way, would you board a plane if you knew the code-coverage figure was less than 50%? Yet we run our mission-critical applications on software with much worse code coverage.

Code-coverage is just but one metric, but it does give us an insight just how tested some components are. So taking that as a data point and marrying it with the fact that developers on the whole don't like writing tests, yet everyone feels a lot safer if they are written we have to look at bridging the gap so everyone wins.

One approach is to automate some of the unit production tests, which can take us so far. Using tools such as TestGen4J we can at least do a cursory set of tests to get us so far down the road. But really what we need to do is to run the application in a simulated world and observe which code gets exercised and how it handles under certain conditions.

This allows us to age the software in much the same way Nagu discusses the Toyota example of how their products are continually feeding them data points even throughout their life cycle. In web environments, playing back simulated worlds is relatively easy by simply reading the access logs and feeding them back through the system.

The main advantage of simulated worlds is they are infinitely more relevant than contrived unit tests that are setup and teared down on each execution run.

Testing is something that shouldn't be taken for granted, and nor should we be lulling ourselves into a false of security through our multitude of unit tests. Unit tests gets us so far down the road, but they do not complete the journey.

There is nothing like the field to really test how something handles; which is why I guess Toyota (and other auto manufacturers) makes use of race tracks so much.


Recent Cloud posts

Recent JAVA posts

Latest CFML posts

Site Links