Audit Testing

Abstract

An automated software testing technique is presented which spares us from having to stipulate our expectations in test code, and from having to go fixing test code each time our expectations change.

Artificial Code Coverage

Abstract

In this paper I put forth the proposition that contrary to popular belief, 100% code coverage can be a very advantageous thing to have, and I discuss a technique for achieving it without excessive effort.

On Mock Objects and Mocking

Abstract

The practice of using Mock Objects in automated software testing is examined from a critical point of view and found to be highly problematic. Opinions of some well known industry speakers are cited. The supposed benefits of Mock Objects are shown to be either no real benefits, or achievable via alternative means.

Introduction

Collaboration Monitoring

An automated software testing technique is presented which solves the fragile test problem of white-box testing by allowing us to ensure that the component-under-test interacts with its collaborators according to our expectations without having to stipulate our expectations as test code, without having the tests fail each time our expectations change, and without having to go fixing test code each time this happens.

On Test-Driven Development (TDD)

Let me get one thing out of the way first: I am open to Test-Driven Development (TDD). I am not currently practicing it, because when I gave it a try some time ago it did not seem to work for me, but I do not have any objections to it in principle, so I might give it another try in the future. Let us just say that it was not love at first sight, but then again some relationships do take some time to warm up.

Having said that, let me now express a few reasons why I am skeptical of TDD. The previous paragraph should have established that I am not trashing TDD, I am just expressing some reservations.

Incremental Integration Testing

A new method for Automated Software Testing is presented as an alternative to Unit Testing. The new method retains the benefit of Unit Testing, which is Defect Localization, but eliminates the need for mocking, thus greatly lessening the effort of writing and maintaining tests.

Software Testing with Fakes instead of Mocks

What are fakes, what are their benefits, and why they are uncontestably preferable over mocks. Also, how to create fakes if needed.

Introduction

White-Box vs. Black-Box Testing

I have something blasphemous to tell you.

Unit Testing is wrong.

There, I said it.

I know I just insulted most people's sacred cow.

Sorry, not sorry.

I will explain, bear with me.

The case for software testing

What to reply to a non-programmer who thinks that testing is unnecessary or secondary

At some point during his or her career, a programmer might come across the following argument, presented by some colleague, partner, or decision maker:

Since we can always test our software by hand, we do not need to implement Automated Software Testing.

Apparently, I reached that point in my career, so now I need to debate this argument. I decided to be a good internet citizen and publish my thoughts. So, in this post I am going to be deconstructing that argument, and demolishing it from every angle that it can be examined. I will be doing so using language that is easy to process by people from outside of our discipline.

In the particular company where that argument was brought forth, there exist mitigating factors which are specific to the product, the customers, and the type of relationship we have with them, all of which make the argument not as unreasonable as it may sound when taken out of context. Even in light of these factors, the argument still deserves to be blown out of the water, but I will not be bothering the reader with the specific situation of this company, so as to ensure that the discussion is applicable to software development in general.

In its more complete form, the argument may go like this:

GitHub project: mikenakis-testana

A command-line utility for running tests by order of dependency and for only running tests that actually need to run.

The mikenakis-testana logo, profile of a crash test dummy by Mike Nakis

based on original work by Wes Breazell, Alexander Skowalsky / Alex Tai, and Jan Pleva.

Used under CC BY License.

On JUnit's random order of test method execution

This is a rant about JUnit, or more precisely, a rant about JUnit's inability to execute test methods in natural method order. 

Definition: Natural method order is the order in which methods appear in the source file.

What is the problem?