Everyone speaks of Unit Testing, but how is it defined? Obviously, it is some kind of software testing, but exactly what is a "unit" supposed to be? And why is it called a "unit"?
Wikipedia
Let us begin by checking the Wikipedia entry for Unit Testing:
Unit testing, a.k.a. component or module testing, is a form of software testing by which isolated source code is tested to validate expected behavior.
Unit testing describes tests that are run at the unit-level to contrast testing at the integration or system level.
Further down in the history section, Wikipedia lists some of the earliest known efforts of what we would today call unit testing, where the common theme is testing separately smaller parts of large software systems before integrating them together.
I am in full agreement with Wikipedia's definition, but Wikipedia is everyone's favorite source to cite if it agrees with their preconceptions, or proclaim untrustworthy if it does not, so can we find any other definition that corroborates Wikipedia's definition?
IEEE
Let us check what IEEE 1008-1987 "Standard for Software Unit Testing" has to say. In the Definitions section we read:
[Warning! wooden language ahead!]
test unit3: A set of one or more computer program modules together with associated control data, (for example, tables), usage procedures, and operating procedures that satisfy the following conditions:
1) All modules are from a single computer program
2) At least one of the new or changed modules in the set has not completed the unit test4
3) The set of modules together with its associated data and procedures are the sole object of a testing process
And the footnotes:
3 A test unit may occur at any level of the design hierarchy from a single module to a complete program. Therefore, a test unit may be a module, a few modules, or a complete computer program along with associated data and procedures.
4 A test unit may contain one or more modules that have already been unit tested.
As we can see, IEEE's definition says nothing about isolated source code; instead, it considers an entire set of modules, of which only one might need testing, as a unit. So, we have found a source that seems to contradict Wikipedia. It is a tie. Now we need to find a third opinion, to achieve a majority vote.
Kent Beck
Surely, Kent Beck, the inventor of Test-Driven Development and author of JUnit must have defined the term, right? Well, as it turns out, no.
In his "Simple Smalltalk Testing: With Patterns" paper (by Kent Beck, First Class Software, Inc. KentBeck@compuserve.com) the closest he gets to providing a definition is the following sentence:
I recommend that developers write their own unit tests, one per class.
Clearly, "one test per class" cannot possibly be regarded as a definition of the term; besides, it does not even match reality.
In Test Driven Development by Example (2002) the closest that Kent Beck gets to providing a definition is the following sentence:
The problem with driving development with small scale tests (I call them “unit tests”, but they don’t match the accepted definition of unit tests very well) is that you run the risk [...]
So, Kent Beck seems to regard unit tests as small-scale tests, which is not really a definition, and he acknowledges that there exists some other, accepted definition, but he does not say what that definition is. Perhaps Kent Beck thinks of a unit test as a unit of testing, as in a unit of information or a unit of improvement, but we cannot be sure.
It is worth noting that Kent Beck does not use mocks. In the video Thoughtworks - TW Hangouts: Is TDD dead? (youtube, text digest)
at 21':10'' Kent Beck states "My personal practice is I mock almost
nothing."
Martin Fowler
One often-cited author who is known for having described many concepts and defined many terms is Martin Fowler. So, what does he have to say about unit testing?
Martin Fowler's page on Unit Test begins by acknowledging that it is an ill-defined term, and that the only characteristics of unit testing that people seem to agree on are that they are supposed to be a) small-scale, b) written by the programmers themselves, and c) fast. Then, Martin Fowler proceeds to talk about two schools of thought that understand the term differently:
- The "classicist" school of thought, which favors "sociable" unit tests, places emphasis on testing the behavior of a component, allowing the component to interact with its collaborators and assuming that the collaborators are working correctly.
- The "mockist" school of thought, which favors "solitary" unit tests, insists on testing each component in isolation from its collaborators, and therefore requires that every collaborator must be replaced with a "test double" for the purpose of testing.
Okay, so this did not lead us to a single definition of unit testing, but at least it helped us further define two competing definitions.
It is also worth noting that Martin Fowler does not use mocks, either. In the video Thoughtworks - TW Hangouts: Is TDD dead? (youtube, text digest) at 23':56'' Martin Fowler adds "I'm with Kent, I hardly
ever use mocks."
Glenford Myers
Let us find the book in which the term "Unit Test" was first introduced, shall
we?
I do not know for sure that the highly acclaimed 1979 classic The Art of Software Testing by Glenford Myers contains the first recorded use of the term, but it is so old that it seems reasonable to suppose that it does.
The term "unit" may have appeared in the original 1979 edition (ISBN 9780471043287, 0471043281) or it may have been added after the fact in the 2nd edition, published in 2004. Unfortunately, the 1979 edition is not easy to obtain, so I cannot ascertain this, but I think it is safe to assume that when Glenford Myers spoke of "Module Testing" he meant what we now call "Unit Testing". Here is what the 2nd edition says:
In chapter 5 "Module (Unit) Testing" we read:
Module testing (or unit testing) is a process of testing the individual subprograms, subroutines, or procedures in a program. That is, rather than initially testing the program as a whole, testing is first focused on the smaller building blocks of the program.
So, this definition is more in line with Wikipedia's definition.
Furthermore, later in the same chapter Glenford Myers states that Unit Testing is a form of white-box testing, and even lays the foundations of what later became mocks:
[...] since module B calls module E, something must be present to receive control when B calls E. A stub module, a special module given the name “E” that must be coded to simulate the function of module E, accomplishes this.
Ian Cooper
Another conflicting opinion comes from Ian Cooper, an outspoken TDD advocate.
In TDD, Where Did It All Go Wrong (2017) (InfoQ, YouTube) Ian Cooper states that in TDD a Unit Test is defined as a test that runs in isolation from other tests, not a test that isolates the unit under test from other units. In other words, the unit of isolation is the test, not the unit under test.
It is worth noting that Ian Cooper also belongs to the ranks of outspoken developers who do not use mocks. In the same talk, at 49':32'' he says "I argue quite heavily against mocks because they are over-specified."
Conclusion
Although not unanimous, the prevailing opinion seems to be that the term unit refers to the component under test, which is supposed to be tested in isolation from its collaborators, in contrast to integration testing and end-to-end testing (a.k.a. system testing) where components are allowed to interact with their collaborators.
A lot of the confusion seems to stem from the fact that Unit Testing requires mocking the collaborators, but most of the people cited in this research realize that the use of mocks is misguided, so they try to give alternative definitions of the term so that they can still call what they do unit testing, even though it really is not.
Cover image: Created by michael.gr using ChatGPT, and then retouched to remove imperfections. The prompt used was: "Please give me an image of a crash test dummy in the style of The Thinker, by Auguste Rodin."
No comments:
Post a Comment