2018-02-05

Mike Nakis on Code Craftsmanship

In a recent job interview I was asked what are my favorite means of ensuring the quality of the code that I write. Off the top of my head I could give a few answers, but it occurred to me afterwards that I could of course have said a lot more. I will try to make a list here.

Please note that in this list I try to avoid repeating things that are common practice, or common knowledge from well read books.  So, for example, I will not mention "Use inversion of control" here, it goes without saying.  I will try to say things that might not be common knowledge, or that might even be controversial.  (Note: "QQ" means "cry me a river.")

When I work for an employer I follow the practices of the house, but when I write software for myself, I tend to do the following:
  • Assert everything.  When I look at code, I don't ask myself "should I assert that?" Instead, I ask myself "is there anything that I forgot to assert?"  The idea is to assert everything that could possibly be asserted, leave nothing unasserted. Assertions take care of white-box testing your code, so software testing can then be confined to the realm of strictly black-box testing, as it should.  Assertions cost nothing, so you can go wild with them.  Go ahead and assert that your array is sorted before attempting to perform binary search on it! Verify that your binary search worked correctly by comparing its result with the result of a linear search for the same item! When I say assert everything, I really mean everything.
  • Do black-box testing, not white-box testing. Heed the advice that says test against the interface, not the implementation. Unit Testing is testing against the implementation, so it should be avoided. Do Incremental Integration Testing instead, which only tests interfaces. With code that is choke-full of assertions, it works really well. Incidentally, this means that mocking, despite being an admirably nifty trick, should for the most part be unnecessary: if you have to resort to using mocks in your tests, then many chances are that a) you have not designed something well, or b) you are doing white-box testing. (And if you have to do ungodly hacks like mocking final/non-virtual or static methods, then you clearly have a wrong design in your hands.)
  • Minimize state, maximize immutability. Design so that as much code as possible is dealing with data that is immutable. Eschew technologies, frameworks, and techniques that prevent or hinder immutability. If you are using auto-wiring, use constructor injection only, so that you can store in final/readonly members.
  • Minimize flow control statements, especially the `if` statement. If there is any opportunity to design something so as to save some `if` statements, the opportunity should be pursued tenaciously.
  • Move the complexity to the design, not the code. If the code does not look so simple that even an idiot can understand it, this usually means that shortcuts have been taken in the design, which have to be compensated for with overly complex code. Make the design as elaborate as necessary so that the code can be as simple as possible. Overly complex code is usually the result of violations of the Single Responsibility Principle.
  • Adhere to the Single Responsibility Principle like your life depends on it. Often, what you think of as a single responsibility can in fact be further sub-divided into more fundamental responsibilities. Almost all of the code that we write performs, or can be thought of as performing, some kind of transformation. Most transformations are of the simplest kind, converting just one type of entity into another, meaning that they involve only two participants. In some cases we have transformations that involve three participants, for example converting one kind of entity into another by consulting yet a third kind of entity, and they tend to be appreciably complex. Four or more participants in a single transformation invariably belong to the realm of the grotesquely complex and must be avoided at all costs. The beautiful thing is that when trying to determine whether you are violating the Single Responsibility Principle, all you need to do is count the number of participants, and that all transformations, no matter how complex, can be refactored into multiple successive transformations of no more than 3 participants each, introducing intermediate kinds of participants if necessary. 
  • Refactor at the slightest indication that refactoring is due; do not allow technical debt to accumulate. Avoid the situation of being too busy mopping the floor to turn off the faucet.  Allow a percentage of sprints to explicitly handle nothing but technical debt elimination. Do not try to spread the task of refactoring over feature development sprints, because a) doing so will not make the refactoring effort magically disappear, b) you will not do a good enough job at it, and c) the time estimation of the features will suffer. If you are dealing with a project manager who fails to see where is the "customer value" in refactoring, quit that job, find another one. 
  • Strive for abstraction and generalization. The urge to abstract and generalize is often mistaken as having reusability as its sole aim, and so it is often met with the YAGNI objection: "You Ain't Gonna Need It". The objection is useful to keep in mind so as to avoid over-engineering, but at the same time it must not be followed blindly, because abstraction and generalization have inherent benefits regardless of the promise of reusability. Every problem of a certain complexity and above, no matter how application-specific it seems to be, can benefit from being divided into an abstract, general-purpose part, and a specialized, application-specific part. Strive to look for such divisions and realize them in the design. The application code will be easier to understand because it will be free from incidental complexity. The general purpose code will be easier to understand because it will be implementing an abstraction.
  • Use domain-specific interfaces. Encapsulate third party libraries behind interfaces of your own devise, tailored to your specific application domain. Strive to make it so that any third-party library can be swapped with another product without you having to rewrite application logic. Conventional wisdom says the opposite: we have all heard arguments like "the best code is the code you don't write" (makes me want to invest in the business of not writing software) or that "a third-party library will be better documented than your stuff" (presumably because documentation is a skill your developers have not mastered) or that "if you run into trouble with a library, you can ask for help on stackoverflow, while if you run into trouble with something you have developed in-house, you are stuck" (presumably because your developers know nothing of it, despite working with it every day.) The truth with application development is that the more you isolate the application logic from peripheral technologies, the more resilient your application logic becomes to the ever changing technological landscape, a considerable part of which is nothing but ephemeral fashions, the use of which is dictated not by actual technological merit, but by C.V. Driven Development instead.
    (See https://martinjeeblog.com/2015/03/11/cv-driven-development-cdd/)
  • Strive for what is simple, not for what looks easy.  The simple often coincides with the easy, but sometimes the two are at odds with each other. Eschew languages and frameworks that provide the illusion of easiness at the expense of simplicity. The fact that a particular framework makes "hello, world!" an easy one-liner probably means that the ten-thousand-liner that you are aiming for will be unnecessarily complicated and hard to write.
    Watch this: https://www.infoq.com/presentations/Simple-Made-Easy
  • Avoid binding by name like the plague. Avoid as much as possible mechanisms whose modus operandi is binding by name: use them only for interfacing with external entities, never for communication between your own modules. Yes, this includes the use of REST. QQ.
  • Always use strong (static) typing. Avoid any kind of weak typing (euphemistically called dynamic typing, duck typing, etc) and avoid languages and frameworks that require it or even just sympathize with it. Yes, this includes pretty much all scripting languages. QQ.
  • Strive for debuggability. For example, do not overdo it with the so-called "fluent" style of invocations, because they are not particularly debuggable.  Do not hurry to adopt this cool new language before you have made sure that debugger support for it is complete and working properly. 
  • Strive for testability.  Design interfaces that expose all functionality that makes sense to expose, not only functionality that is known to be needed by the code that will invoke them. For example, the application may only need an interface to expose a `register()` and `unregister()` pair of methods, but `isRegistered()` also makes sense to expose, and it will incidentally facilitate (black-box) testing. (This is a trivial example, hopefully you see the bigger picture.)
  • Enable all warnings that can possibly be enabled. The fact that a certain warning may, on rare occasions, be issued on legitimate code, is no reason to disable the warning. The warning should be enabled, and selectively suppressed on a case by case basis. Some warnings, like "unused identifier", occur on legitimate code too often for selective suppression to be practical. For those warnings, consider using an IDE that supports a "weak warning" level, which is highlighted inconspicuously, so it can be easily filtered out by your eyes, but the visual clue is there in case it points to something unexpected. And of course some silly warnings occur on legitimate code all the time, so it goes without saying that they need to be disabled.
  • Strive for readability. Code is generally write-once, read many. We tend to read our code several times as we write it, and then many more times throughout its lifetime as we tweak it, as we write nearby code, as we browse through code to understand how things work, as we perform troubleshooting, etc. Therefore, choices that make code easier to read are preferable even if they make code a bit harder to write. This means that languages whose primary claim to fame is terseness of code are not really delivering something of value, because verbosity of code is not one of the major problems that our profession is faced with; unreadable code is. This also means that certain languages whose grotesquely arcane syntax has earned them the "write-only language" designation (I am looking at you, perl) are not to be touched even with a 10 ft. pole. QQ.
  • Use an IDE with a spell checker. Avoid acronyms and abbreviations, and anything that fails to pass the spell check. Modern IDEs have formidable auto-completion features, so using long identifiers does not mean that you have to type more. But even if it did, typing is not one of the major problems that our profession is faced with; unreadable code is.
  • Pay attention to naming. Strive for good identifier names and for a variety of names that reflect the variety of the concepts. Any piece of code written by a programmer whose English language skills are poor should be reviewed by a programmer whose English language skills are good. A Thesaurus is an indispensable programming tool. Spend the necessary time to find the right word to name something, and dare to use names that you may have never heard anyone using before. For example, if you are wondering how to call a Collection of Factories, why not call it Industry?
  • Code offensively, not defensively.  This means never fail silently, never allow any slack or leeway, keep tolerances down to absolute zero. Fail fast, fail hard, fail eagerly and enthusiastically. Avoid things like a `Map.put()` method which either adds or replaces, and instead design for `add()` methods which assert that the item being added does not already exist, and `replace()` methods which assert that the item being replaced does in fact already exist. If an add-or-replace operation is useful, (and it very rarely is,) give it a name that clearly indicates the weirdness in what it does: call it `addOrReplace()`. (Duh!) Similarly, avoid things like a `close()` method which may be invoked more than once with no penalty: assert that your `close()` methods are invoked exactly once. If you are unsure just how many times your code might invoke your `close()` method, you have greater problems to worry about than an assertion failing inside your `close()` method.  Read this: http://trevorjim.com/postels-law-is-not-for-you
  • Use inheritance when it is clearly the right choice. The advice that composition should be favored over inheritance was very good advice during the nineties, because back then people were overdoing it with inheritance: the general practice was to not even consider composition unless all attempts to first get things to work with inheritance failed. That practice was bad, and the fact that the predominant language at that time (C++) supported not just inheritance but actually multiple inheritance made things even worse. So the advice against that practice was very much needed. However, the advice is still being religiously followed to this day, as if inheritance had always been a bad thing. This is leading to unnecessarily convoluted designs and much weeping and gnashing of teeth. Even the original advice suggested favoring one over the other, it did not prescribe the complete abolition of the other. So, today it is about time we reword the advice to read know when to use inheritance and when to use composition.
  • Favor early exits over deep nesting. This means liberal use of the `break` and `continue` keywords, as well as `return` statements in the middle of a method. The code ends up being a lot simpler this way. Yes, this directly contradicts the ancient "one return statement per function" dogma.  It is nice to contradict ancient dogma.
  • Avoid static mutable state as much as possible. Yes, this also includes stateful singletons. The fact that it only makes logical sense to have a single instance of a certain one-of-a-kind object in your world is no reason to design that object so that only one instance of it can ever be. You see, I guarantee to you that the need will arise in the future, unbeknownst to you today, to multiply instantiate your world, with that object in it which you thought was one of a kind.
  • Put the tools of the trade into use.  Armies of very good developers have worked hard to build these tools, don't you dare make their efforts go in vain. 
    • Use an IDE. Programmers who think that they are better off with their favorite text editor and their favorite assortment of command line tools should be admitted to rehabilitation.  That having been said, also be weary of programmers who are so attached to their IDE that they program by dragging and dropping code snippets around.  Perhaps some desktop publishing job would better suit them.
    • Use your IDE for building and running tests.  Programmers who habitually build and test using maven-install should be fired, programmers who habitually build and test using maven-clean-install should be shot. The information we painstakingly record into `pom.xml` files is for maven to use when launched by Continuous-Integration, and for the IDE to parse in order to rebuild its own project files.  In our daily work we use the IDE, not maven.
    • Use the debugger of your IDE as your first choice for troubleshooting anything, not as the last resort after all other options have been exhausted. This means that you should not be using the debugger only when there is trouble, you should be using it always, by default, so that it is ready when trouble occurs. This in turn means that you should never hit the "run" key on your IDE; hit the "debug" key instead. Always the "debug" key. Only the "debug" key. Configure your IDE so that the debugger pops up when an exception is thrown, instead of relying on examination of postmortem stack traces in the logs. Stack traces are for troubleshooting problems in production, and it is best if it never comes to that.
    • Do not optimize anything unless:
      • You know beyond any doubt that there is in fact a performance problem, and 
      • The profiling tool has shown precisely where the problem is. 
    • Do not even think that you are done with testing unless the code coverage tool gives you sufficient reason to suspect so. 
    • Have your IDE perform code analysis on commit, and incorporate even more code analysis in the nightly or continuous build.
  • Design with reliability as a foundation, not as an afterthought.  For example, sharing data in a multi-threaded environment by means of traditional locking ("synchronization") techniques is error-prone and untestable, because you cannot test for race conditions. Therefore, these techniques of sharing data must be abandoned. Instead, design for a lock-free, share-nothing approach that works by passing immutable messages, thus eliminating the very possibility of race conditions.
  • Design with security as a foundation, not as an afterthought.  Security is not something that you can add on top of an insecure foundation, because there is no amount of carefulness on behalf of the developers that is careful enough, and no kind of automated testing that can detect security hazards. So, what is necessary is architectural choices that eliminate the very possibility of entire classes of security hazards. (Do not worry, there will always be other classes of security hazards to have to worry about.) If a certain architectural choice is prone to vulnerabilities, do not make that choice. An example of a vulnerability-prone architectural choice which should be avoided like anthrax is putting application code on the web browser, otherwise known as "full-stack web development". QQ.
  • Keep the logs clean. Do not vex your colleagues, and do not make your own life harder, with torrential info-level or debug-level spam in the logs. Keep the info-level messages down to an absolute minimum, and once debugging is done, completely remove all the debug-level log statements. Regularly use the "blame" feature of the version control system to remind developers of logging statements that they should remove. Never use the log for capturing metrics or any other kind of structured information; use some separate, specialized instrumentation recording facility for that.
  • Take maxims with a grain of salt. When someone says "no function should ever accept more than 4 parameters" or "no class should ever be longer than 250 lines" they are usually talking nonsense. A function should accept as many parameters as necessary to do its job, and if that is 15 parameters, so be it. A class should be as long as necessary to do its job, and if that is 2000 lines, so be it. Breaking things down to smaller units should be done because there is some actual merit in doing so, not because some prophecy said so.
  • Private static methods are fine. Really. Instance methods have the entire state of their object at their disposal to read and manipulate, and this state may be altered by each and every other instance method in the entire object. Static methods, on the other hand, are obviously not in a position to read nor alter any of the object's state, and instead rely exclusively on parameters and return values, which are all clearly visible at each call site. Thus, static methods are magnificently less complex beasts than instance methods. I am not saying that you should strive to put as much of your code as possible in private static methods, (although a case could be made even for that,) but what I am saying is that private static methods are not the slightest bit evil as some folks think they are.  They are good.  We should have more of them.
  • Do not fix it unless there is a test for it. So far I have not tried test-driven development, so I do not have an opinion about it yet, but what I have tried, and I have found to be immensely useful, is test-driven maintenance. So, if someone discovers a bug, which obviously passed whatever automated tests you already had in place, do not hurry to fix it. First, write a test that tests against the bug and fails. Then, fix the bug and watch the test pass. And hopefully you have enough tests in place for every other part of your software system so as to have reasonable guarantees that in fixing this bug you did not break anything else.
  • Use the type system to the fullest. Try to avoid using general purpose data types; try to use data types that are specific to the job instead. A classic example of this is the suggestion to always use a `Duration` data type instead of an `int` number of milliseconds, but it goes further than that. So, no, your age and height are not of type `double` or `int`, they are of type `Age` and `Height` respectively. Your married status is not a boolean, it is an instance of `MarriedStatus`. The fact that your `SpiffyStorage` class can accept any string as a key is irrelevant: introduce a `SpiffyKey` data type, and make it contain nothing but a string if need be, but it is a SpiffyKey, not a String. 
  • Avoid death by ten thousand little methods. Again and again I see codebases with multitudes of tiny methods containing just one or two lines of trivial code aiming to ensure that not a single line of code is duplicated anywhere. The downside of this is that it increases the complexity of the calling tree and therefore the amount of mental effort required to traverse it and make sense out of it, or verify its correctness. A new function is worth introducing if it has a well-defined, meaningful role to play. Difficulty in coming up with a name for it is a good indicator that it has no role to play other than to avoid code duplication. Of course there is merit in reducing code duplication, but not when the code in question is trivial. And when you see the possibility to deduplicate non-trivial code, then the well-defined, meaningful role of the function is always readily visible, it calls out to you, and the appropriate name for it is immediately obvious, you don't have to squeeze your head trying to name it, nor do you have to resort to cumbersome names like `common-Spline-Reticulation-Handler-Before-Common-Dive-For-Moog()`.
I have been, and will continue to be, extending this list with the passage of time.

No comments:

Post a Comment