2018-02-05

Mike Nakis on Code Craftsmanship

In a recent job interview I was asked what are my favorite means of ensuring the quality of the code that I write. Off the top of my head I could give a few answers, but it occurred to me afterwards that I could of course have said a lot more. I will try to make a list here.  I could probably write a book on this.



  • Assert everything.  When I look at code, I don't ask myself "should I assert that?" Instead, I ask myself "is there anything that I forgot to assert?"  The idea is to assert everything that could possibly be asserted, leave nothing unasserted. Assertions take care of white-box testing your code, so software testing can then be confined to the realm of strictly black-box testing, as it should.  Assertions cost nothing, so you can go wild with them.  Go ahead and assert that your array is sorted before attempting to perform binary search on it! Verify that your binary search worked correctly by comparing its result with the result of a linear search for the same item! When I say assert everything, I really mean everything.
  • Do black-box testing, not white-box testing. Heed the advice that says test against the interface, not the implementation. Unit Testing is testing against the implementation, so it should be avoided. Do Incremental Integration Testing instead, which only tests interfaces. With code that is choke-full of assertions, it works really well. Incidentally, this means that mocking, despite being an admirably nifty trick, should for the most part be unnecessary: if you have to resort to using mocks in your tests, then many chances are that a) you have not designed something well, or b) you are doing white-box testing.
  • Minimize state, maximize immutability. Design so that as much code as possible is dealing with data that is immutable. Re-examine every single class which contains mutable members, and many chances are you will find that it could be replaced with an immutable class.  Even if not, you might discover that many of its members could become immutable. Eschew any frameworks, technologies, or techniques that prevent or hinder immutability. If you are using auto-wiring, use constructor injection only, so that you can store in final/readonly members. Note, however, that immutability is not important in function-local variables. It is still good practice as it tends to make things more simple, but there is absolutely no problem with having function-local mutation if it serves the slightest purpose. Which brings us to the next point:
  • Do overwrite function parameters. Go ahead and do this when the original parameter value should not be used in the remainder of the function. In doing so you are minimizing the number of variables that are in scope and preventing accidental use of the original value. The origins of the cargo-cult programming convention of avoiding this are purely historical and entirely arcane: some old versions of Fortran would pass everything by reference, including constants, so if someone invoked your function with the constant 3 for parameter x, and then within the function you assigned 5 to parameter x, then from that moment on the constant 3 would stand for 5 in the entire program. As a result, old computer scientists picked up the habit of never reassigning function parameters, and they kept passing this advice to newer generations.
  • Avoid b-to-a style conversions, use a-from-b style instead.  When I see 'a = aFromB( b )' I can immediately tell that it looks correct, while when I see 'b = aToB( a )' I have to stare at it for a little while longer before I can tell whether it is correct or not. (This is related to Joel Spolky's notion that correct code should look correct, and wrong code should look wrong. )
  • Avoid unnecessary braces. Doing so keeps the code more compact, making more statements fit within the screen. The cargo-cult programming convention of enclosing even single-statement controlled blocks within curly braces allegedly avoids bugs caused by trying to add a second statement to the block while forgetting to introduce curly braces. This has actually happened to me once, and the programmer who introduced the bug in my code did not even apologize, he considered it my fault for not having provided the curly braces for him to insert his second statement in. The fact of the matter is that a halfway decent IDE will issue a warning about incorrect indentation of controlled blocks, so this is not a problem today. Of course, in order to have that warning enabled, you must be keeping a consistent indentation style everywhere, right?
  • Minimize flow control statements, especially the `if` statement. If there is any opportunity to structure a piece of code so as to save an `if` statement, the opportunity should be pursued tenaciously.
  • Move the complexity to the design, not the code. If the code does not look so simple that even an idiot can understand it, this usually means that shortcuts have been taken in the design, which have to be compensated for with overly complex code. Make the design as elaborate as necessary so that the code can be as simple as possible. Overly complex code is usually the result of violations of the Single Responsibility Principle.
  • Adhere to the Single Responsibility Principle like your life depends on it. Often, what you think of as a single responsibility can in fact be further sub-divided into more fundamental responsibilities. Almost all of the code that we write performs, or can be thought of as performing, some kind of transformation. Most transformations are of the simplest kind, converting just one type of entity into another, meaning that they involve only two participants. In some cases we have transformations that involve three participants, for example converting one kind of entity into another by consulting yet a third kind of entity, and they tend to be appreciably complex. Four or more participants in a single transformation are grotesquely complex and must be avoided at all costs. Luckily, they can always be refactored into multiple successive transformations of no more than 3 participants each, introducing intermediate kinds of participants if necessary.
  • Refactor at the slightest indication that refactoring is due; do not allow technical debt to accumulate. Avoid the situation of being too busy mopping the floor to turn off the faucet.  Allow a percentage of sprints to explicitly handle nothing but technical debt elimination. Do not try to spread the task of refactoring over feature development sprints, because a) doing so will not make the refactoring effort magically disappear, b) you will not do a good enough job at it, and c) the time estimation of the features will suffer. If you are dealing with a project manager who fails to see where is the "customer value" in refactoring, quit that job, find another. 
  • Strive for abstraction and generalization. The urge to abstract and generalize is often mistaken as having reusability as its sole aim, and so it is often met with the YAGNI objection: "You Ain't Gonna Need It". The objection is useful to keep in mind so as to avoid over-engineering, but at the same time it must not be followed blindly, because abstraction and generalization have inherent benefits regardless of the promise of reusability. Every problem of a certain complexity and above, no matter how application-specific it might seem to be, can benefit from being divided into a specialized, application-specific part and an abstract, general-purpose part. Strive to look for such divisions and realize them in the design. The general purpose code will be easier to understand because it will be implementing an abstraction. The application code will be easier to understand because it will be free from incidental complexity. If you can choose between adding 5 lines of application code vs. adding 2 lines of application code and 10 lines of framework code, opt for the latter, regardless of whether those 10 lines of framework code are likely to ever be reused.
  • Use domain-specific interfaces. Encapsulate third party libraries behind interfaces of your own devise, tailored to your specific application domain. Strive to make it so that any third-party library can be swapped with another product without you having to rewrite application logic. Conventional wisdom says the opposite: we have all heard arguments like "the best code is the code you don't write" (makes me want to invest in the business of not writing software) or that "a third-party library will be better documented than your stuff" (presumably because documentation is a skill your developers have not mastered) or that "if you run into trouble with a library, you can ask for help on stackoverflow, while if you run into trouble with something you have developed in-house, you are stuck" (presumably because your developers know nothing of it, despite working with it every day.) The truth with application development is that the more you isolate the application logic from peripheral technologies, the more resilient your application logic becomes to the ever changing technological landscape, a considerable part of which is nothing but ephemeral fashions, the use of which is dictated not by actual technological merit, but by C.V. Driven Development instead.
    (See https://martinjeeblog.com/2015/03/11/cv-driven-development-cdd/)
  • Strive for what is simple, not for what looks easy.  The simple often coincides with the easy, but sometimes the two are at odds with each other. Eschew languages and frameworks that provide the illusion of easiness at the expense of simplicity. The fact that a particular framework makes "hello, world!" an easy one-liner probably means that the ten-thousand-liner that you are aiming for will be unnecessarily complicated and hard to write.
    Watch this: https://www.infoq.com/presentations/Simple-Made-Easy
  • Avoid binding by name like the plague. Avoid as much as possible mechanisms whose modus operandi is binding by name: use them only for interfacing with external entities, never for communication between your own modules.  REST enthusiasts can cry me a river.
  • Always use strong (static) typing. Avoid any kind of weak typing (euphemistically called dynamic typing, duck typing, etc) and avoid languages and frameworks that require it or even just sympathize with it. Yes, this includes pretty much all scripting languages. Script kiddies can cry me a river.
  • Strive for debuggability. For example, do not overdo it with the so-called "fluent" style of invocations, because they are not particularly debuggable.  Do not hurry to adopt this cool new language before you have made sure that debugger support for it is complete and working properly. 
  • Strive for testability.  Design interfaces that expose all functionality that makes sense to expose, not only functionality that is known to be needed by the code that will invoke them. For example, the application may only need an interface to expose a `register()` and `unregister()` pair of methods, but `isRegistered()` also makes sense to expose, and it will incidentally facilitate (black-box) testing. (This is a trivial example, hopefully you see the bigger picture.)
  • Enable all warnings that can possibly be enabled. The fact that a certain warning may, on rare occasions, be issued on legitimate code, is no reason to disable the warning. The warning should be enabled, and selectively suppressed on a case by case basis. Some warnings, like "unused identifier", occur on legitimate code too often for selective suppression to be practical. For those warnings, consider using an IDE that supports a "weak warning" level, which is highlighted inconspicuously, so it can be easily filtered out by your eyes, but the visual clue is there in case it points to something unexpected. And of course some silly warnings occur on legitimate code all the time, so it goes without saying that they need to be disabled.
  • Strive for readability. Code is generally write-once, read many. We tend to read our code several times as we write it, and then many more times throughout its lifetime as we tweak it, as we write nearby code, as we browse through code to understand how things work, as we perform troubleshooting, etc. Therefore, choices that make code easier to read are preferable even if they make code harder to write. This means that languages whose primary claim to fame is terseness of code are not really delivering something of value, because verbosity of code is not one of the major problems that our profession is faced with; unreadable code is. This also means that certain languages whose grotesquely arcane syntax has earned them the "write-only language" designation (I am looking at you, perl) are not to be touched with a 10 ft. pole. Perl enthusiasts can cry me a river.
  • Use an IDE with a spell checker. Avoid acronyms and abbreviations, and anything that fails to pass the spell check. Modern IDEs have formidable auto-completion features, so using long identifiers does not mean that you have to type more. But even if it did, typing is not one of the major problems that our profession is faced with; unreadable code is.
  • Pay attention to naming. Strive for good identifier names and for a variety of names that reflect the variety of the concepts. Any piece of code written by a programmer whose English language skills are poor should be reviewed by a programmer whose English language skills are good. A Thesaurus is an indispensable programming tool. Spend the necessary time to find the right word to name something, and dare to use names that you may have never heard anyone using before. For example, if you are wondering how to call a Collection of Factories, why not call it Industry?
  • Code offensively, not defensively.  This means never fail silently, never allow any slack or leeway, keep tolerances down to absolute zero. Fail fast, fail hard, fail eagerly and enthusiastically. Avoid things like a `Map.put()` method which either adds or replaces, and instead design for `add()` methods which assert that the item being added does not already exist, and `replace()` methods which assert that the item being replaced does in fact already exist. If an add-or-replace operation is useful, (and it very rarely is,) give it a name that clearly indicates the weirdness in what it does: call it `addOrReplace()`. (Duh!) Similarly, avoid things like a `close()` method which may be invoked more than once with no penalty: assert that your `close()` methods are invoked exactly once. If you are unsure just how many times your code might invoke your `close()` method, you have greater problems to worry about than an assertion failing inside your `close()` method.  Read this: http://trevorjim.com/postels-law-is-not-for-you
  • Use inheritance when it is clearly the right choice. The advice that composition should be favored over inheritance was very good advice during the nineties, because back then people were overdoing it with inheritance: the general practice was to not even consider composition unless all attempts to first get things to work with inheritance failed. That practice was bad, and the fact that the predominant language at that time (C++) supported not just inheritance but actually multiple inheritance made things even worse. So the advice against that practice was very much needed. However, the advice is still being religiously followed to this day, as if inheritance had always been a bad thing. This is leading to unnecessarily convoluted designs and much weeping and gnashing of teeth. Even the original advice suggested favoring one over the other, it did not prescribe the complete abolition of the other. So, today it is about time we reword the advice to read know when to use inheritance and when to use composition.
  • Favor early exits over deep nesting. This means liberal use of the `break` and `continue` keywords, as well as `return` statements in the middle of a method. The code ends up being a lot simpler this way. Yes, this directly contradicts the ancient "one return statement per function" dogma.  It is nice to contradict ancient dogma.
  • Avoid static mutable state like anthrax. Yes, this also includes stateful singletons. The fact that it only makes logical sense to have a single instance of a certain one-of-a-kind object in your world is no reason to design that object so that only one instance of it can ever be. You see, I guarantee to you that the need will arise in the future, unbeknownst to you today, to multiply instantiate your world, with that object in it which you thought was one of a kind.
  • Put the tools of the trade into use.  Armies of very good developers have worked hard to build these tools, don't you dare make their efforts go in vain. 
    • Use an IDE. Programmers who think that they are better off with their favorite text editor and their favorite assortment of command line tools should be admitted to rehabilitation.  That having been said, also be weary of programmers who are so attached to their IDE that they program by dragging and dropping code snippets around.  Perhaps some desktop publishing job would better suit them.
    • Use your IDE for building and running tests.  Programmers who habitually build and test using maven-install should be fired, programmers who habitually build and test using maven-clean-install should be shot. The information we painstakingly record into `pom.xml` files is for maven to use when launched by Continuous-Integration, and for the IDE to parse in order to rebuild its own project files.  In our daily work we use the IDE, not maven.
    • Use the debugger of your IDE as your first choice for troubleshooting anything, not as the last resort after all other options have been exhausted. This means that you should not be using the debugger only when there is trouble, you should be using it always, by default, so that it is ready when trouble occurs. This in turn means that you should never hit the "run" key on your IDE; hit the "debug" key instead. Always the "debug" key. Only the "debug" key.
    • Do not optimize anything unless:
      • You know beyond any doubt that there is in fact a performance problem, and 
      • The profiling tool has shown precisely where the problem is. 
    • Do not even think that you are done with testing unless the code coverage tool gives you sufficient reason to suspect so. 
    • Have your IDE perform code analysis on commit, and incorporate even more code analysis in the nightly or continuous build.
  • Design with reliability as a foundation, not as an afterthought.  For example, sharing data in a multi-threaded environment by means of traditional locking ("synchronization") techniques is error-prone and untestable, because you cannot test for race conditions. Therefore, these techniques of sharing data must be abandoned. Instead, design for a lock-free, share-nothing approach that works by passing immutable messages, thus eliminating the very possibility of race conditions.
  • Design with security as a foundation, not as an afterthought. Security is not something that you can add on top of an insecure foundation, because there exist no automated tests that can detect security hazards and no amount of carefulness on behalf of programmers that is careful enough. So, what is necessary is architectural choices that eliminate the very possibility of entire classes of security hazards. (Do not worry, there will always be other classes of security hazards to have to worry about.) If a certain architectural choice is prone to vulnerabilities, do not make that choice. An example of a vulnerability-prone architectural choice which should be avoided like COVID19 is putting application code on the web browser. Full-stack developers can cry me a river.
  • Keep the logs clean. Do not vex your colleagues, and do not make your own life harder, with torrential info-level or debug-level spam in the logs. Keep the info-level messages down to an absolute minimum, and once debugging is done, completely remove all the debug-level log statements. Regularly use the "blame" feature of the version control system to remind developers of logging statements that they should remove. Never use the log for capturing metrics or any other kind of structured information; use some separate, specialized instrumentation facility for that.
  • Take maxims with a grain of salt. When someone says "no function should ever accept more than 4 parameters" or "no class should ever be longer than 250 lines" they are usually talking nonsense. A function should accept as many parameters as necessary to do its job, and if that is 15 parameters, so be it. A class should be as long as necessary to do its job, and if that is 2000 lines, so be it. Breaking things down to smaller units should be done because there is some actual merit in doing so, not because some prophecy said so.
  • Private static methods are fine. Really. An instance method has the entire object state at its disposal to read and manipulate, and this state may be altered by any other instance method, including instance methods invoked by the current instance method. A static method on the other hand is obviously not in a position to read nor alter any of the object's state, and it is not able to invoke any instance methods that would do that. Instead, by its nature it has to rely exclusively on parameters, which are all clearly visible at each call site. Thus, it is a magnificently less complex beast than an instance method. What this means is that private static methods are not the slightest bit evil as some folks are under the impression that they are, and we should have more of them. Personally, when I have a class that both has complex logic and mutable state I tend to move the complex logic into private static methods, reducing the public instance methods to doing nothing but packing parameters and instance fields into the argument lists of the private static methods that they invoke.
  • Do not fix it unless there is a test for it. So far I have not tried test-driven development, so I do not have an opinion about it yet, but what I have tried, and I have found to be immensely useful, is test-driven maintenance. So, if a bug is discovered, which obviously passed whatever automated tests you already had in place, do not hurry to figure out what the cause is and fix it. First, write a test that tests against the bug and fails, being completely agnostic of any theory that you might already have in your mind as to what is causing the bug. Then, fix the bug and watch the test pass. (And hopefully you have enough tests in place for every other part of your software system so as to have reasonable guarantees that in fixing this bug you did not break anything else.)
  • Use the type system to the fullest. Try to avoid using general purpose data types; try to use data types that are specific to the job instead. A classic example of this is the suggestion to always use a `Duration` data type instead of an `int` number of milliseconds, but it goes further than that. So, no, your height is not of type `double`, it is of type `Length`. Your married status is not a boolean, it is an instance of `MarriedStatus`. The fact that your `SpiffyStorage` class can accept any string as a key is irrelevant: introduce a `SpiffyKey` data type, and make it contain nothing but a string if need be, but it is a SpiffyKey, not a String. 
  • Avoid death by ten thousand little methods. Again and again I see codebases with multitudes of tiny methods containing just one or two lines of trivial code aiming to ensure that not a single line of code is duplicated anywhere. The downside of this is that it increases the complexity of the calling tree and therefore the amount of mental effort required to make sense out of it. A new function is worth introducing if it has a well-defined, meaningful role to play. Difficulty in coming up with a name for a function, or having many functions with names that differ only slightly and do not readily convey the difference between them, are both good indicators that these functions have no role to play other than to avoid code duplication. Of course there is merit in reducing code duplication, but not when the code in question is trivial. And when you see the possibility to deduplicate non-trivial code, then the well-defined, meaningful role of the function as well as the appropriate name for it tend to be immediately obvious.
  • Make the best out of break-on-exception. Some old programmers who learned to code without debuggers have the bad habit of doing all their troubleshooting by examining logs and post-mortem stack traces and theorizing as to what went wrong, instead of having the debugger break on exception and actually seeing what went wrong. Unfortunately, exceptions are a somewhat complex topic, and debuggers have correspondingly complex mechanisms for dealing with them, so many people do not know exactly what to do. Here is a little primer.
    • Case 1: Exceptions thrown by code which is not your own and you cannot afford to be bothered with. You can take care of them by configuring your debugger either to ignore exceptions thrown in other modules or to only break on exceptions thrown in your modules. Hopefully, the debugger is smart enough to treat an exception which is thrown in another module but propagates into your code as an exception which is thrown in your module; if not, you are stuck, find a better debugger.
    • Case 2: Exceptions thrown within your code.
      • Case 2.1: Exceptions thrown within your code which are never okay to throw, i.e. "Dereference of Null Pointer". For those, you want to configure the debugger to always break, regardless of whether they are handled or unhandled. 
      • Case 2.2: Exceptions thrown within your code which can be okay to throw, and are in fact being thrown and caught by your code, so you do not want the debugger to break, e.g. a "File Sharing Violation", which is usually handled by retrying the operation after a short delay.  All these exceptions can be taken care of simply by configuring the debugger to only break on unhandled exceptions.
      • Case 2.3: Exceptions thrown within your code which can be okay to throw, but are being thrown under unexpected circumstances, so you would like the debugger to break, in order to troubleshoot, but the debugger does not break because all of your exceptions are essentially handled because you have the roots of all your call trees enclosed within try-catch blocks so as to never allow an exception to propagate to the operating system unhandled. To circumvent this problem, make those root try-catch blocks conditional based on whether this is a debug run or not. This way, unexpected exceptions will not be considered as handled. Note that in the Microsoft world a debug run is when the debugger is attached, while in the java world a debug run essentially is when assertions are enabled.
  • Make the best out of the log. You should at all times be able to click on a log entry and be taken to the source line that generated the log entry. I am appalled by how many programming environments do not have this as their default mode of operation.
    • In the Microsoft Visual Studio world, this is achieved by prefixing each log line with the pathname of the source file, followed by an opening parenthesis, followed by a line number, followed by a closing parenthesis and a colon.  Luckily, both C++ and C# support efficient means of obtaining source file name and line number information: In C++ it is the __FILE__ and __LINE__ built-in pre-processor macros, while in C# it is the CallerFilePath and CallerLineNumber attributes. Unfortunately, the pathnames generated by these mechanisms are absolute, meaning that they start from the drive letter and include the kitchen sink, so you might want to programmatically convert them to pathnames relative to the solution folder before logging them. Visual studio also recognizes those, although this behavior might be undocumented and therefore subject to change.
    • In the JetBrains IntelliJ IDEA world, this is achieved using a hacky and entirely undocumented mechanism which is of course subject to change without notice, but as of the time of this writing it involves including anywhere within a log entry a package name followed by an opening parenthesis, followed by a source filename-plus-extension, (but no path,) followed by a colon, followed by a line number, followed by a closing parenthesis. The package name can actually be anything, Intellij expects an identifier there but does not do anything meaningful with it. Note that if the word "at" appears in the log line, and if it appears in any place other than immediately before the package name, then this mechanism breaks. Also note that this mechanism suffers from ambiguity in the case of multiple source files with the same filename. An alternative mechanism is to include a "file://" URI in the log entry, but then you would have to figure out the path from the package name, which might not be easy. Unfortunately, Java does not provide any efficient means of obtaining source file name and line number information, so one has to internally throw an exception and obtain stack trace information from it in order to figure out the source file name and line number of the code that issued the log entry. Fortunately, throwing an exception in the java world is not anywhere near as expensive as in the Microsoft world. Unfortunately, it is still expensive. You can see this performance penalty as one more reason to keep logging to a minimum.

No comments:

Post a Comment