2022-05-02

Deep immutability assessment

Programmers all over the world are embracing immutability more and more; however, mutation is still a thing, and in all likelihood it will continue being a thing for as long as there will be programmers. In a world where both mutable and immutable objects exist side by side, there is often a need to ascertain that a certain object is of the immutable variety before proceeding to use it for certain purposes. 

For example, when an object is used as a key in a hash map, the object better be immutable, or else the hash code of the key may change, causing the map to severely malfunction. Note that when this happens, it tends to be a bug which is very difficult to troubleshoot.

Unfortunately, immutability assessment is not an easy task. Most don't even consider it, few talk about it, even fewer actually do it. Programmers all over the world are accustomed to using objects in scenarios where immutability is required, but without ascertaining it, essentially praying that the objects be immutable. There exist libraries that will ascertain immutability, but judging by how marginal status these libraries have in the greater technology landscape, they are not being put into much use.

Introducing Bathyscaphe

Bathyscaphe is a very small library aiming to give the Java world another chance at addressing the problem of immutability assessment instead of letting it linger on like a chronic ailment that you really should go see a doctor about, but keep postponing because you have no time for that.

Bathyscaphe consists of 4 modules: 

  • bathyscaphe-claims contains annotations that you can add to your classes to aid immutability assessment in certain cases. For example, when a field is not declared as final, but you want to promise that it will behave as if it was final, you can annotate the field with `@Invariable.` Most client code is expected to make use of only this module of bathyscaphe, and the jar is only a couple of kilobytes, since it contains no code, only definitions.
  • bathyscaphe is the core immutability assessment library. A software system will probably invoke this library in only a few places, where immutability needs to be ascertained. The jar is only about 100 kilobytes.
  • bathyscaphe-print is a diagnostic aid which can be used to generate detailed human-readable text explaining precisely why a particular assessment was issued, in the event that an object which you intended to be immutable turns out to be mutable, and you want to know why this happened.
  • bathyscaphe-test is, of course, the tests, which are extensive and achieve close to 100% coverage.

With the exception of the test module, which depends on JUnit, Bathyscaphe does not have any dependencies besides the Java Runtime Environment. Let me repeat this: Bathyscaphe. Has. No. Dependencies. It depends on nothing. When you include Bathyscaphe in a project, you are including Bathyscaphe and nothing else.

(Actually, at the moment that these lines are written, Bathyscaphe does have one external dependency, which is a 1KB jar used for debugging, but this will be removed in the near future.)

How it works

Oftentimes we can tell whether an object is mutable or immutable just by examining its class: some classes can be conclusively assessed as mutable, while some classes can be conclusively assessed as immutable. There exist static analysis tools that can determine this. However, in many cases it is not enough to just examine the class; instead, it is necessary to examine each and every instance of that class at runtime. Static analysis tools that nonetheless try to handle such cases tend to issue assessments that are wrong, or at best useless. For example:

  • Effectively immutable classes behave in a perfectly immutable fashion, but under the hood they are strictly speaking mutable, due to various reasons, for example because they perform lazy initialization, or because they contain arrays, which are by definition mutable. The most famous example of such a class is `java.lang.String`.
    • Static analysis tools tend to erroneously classify effectively immutable classes as mutable, which is a false negative.
    • Note that `java.lang.String` is not so much of a problem, because it can be treated as a special case, but special-casing is a drastic measure which should be used as seldom as possible, and certainly not for every single effectively-immutable class that we write.
  • Superficially immutable classes are classes which are unmodifiable, but they contain members whose immutability they cannot vouch for. The most famous example of classes in this category are the so-called unmodifiable collection classes of java, such as the result of invoking `java.util.List.of()`.
    • Some static analysis tools erroneously report superficially immutable classes as immutable, which is a false positive, as we can easily prove with `List.of( new StringBuilder() )`.
    • Some static analysis tools erroneously report superficially immutable classes as mutable, which is a false negative, as we can easily prove with `List.of( 1, 2, 3 )`.
    • So, to the question "is the class returned by List.of() immutable?" the only correct answer is "I don't know". We cannot issue a conclusive immutability assessment just by looking at the class, we need to perform deep immutability assessment on each and every instance of that class. That's what Bathyscaphe does. And that's why it is called Bathyscaphe.

When trying to assess whether an object is immutable or not, Bathyscaphe begins by looking at the class of the object. For any given class, Bathyscaphe issues one of three possible assessments:

  • Mutable
  • Immutable
  • Provisory

The first two are straightforward: if the class of an object can be conclusively assessed as mutable or immutable, then each instance of that class immediately receives the same assessment, and we are done. However, if the class of an object has been assessed as provisory, this means that the object might be immutable, but some runtime characteristics of the object need to be thoroughly examined. Bathyscaphe performs this examination, and issues an assessment for an object as either mutable or immutable.

For example, if a class looks immutable in all aspects except that it declares a final field of interface type, Bathyscaphe will assess that class as provisory, with the provision being that on instances of that class the value of that field must be checked. If the value of the field turns out to be mutable, then the containing instance is mutable; if the value is immutable, then the containing instance is immutable.

Where to find it

Bathyscaphe is hosted on github; see https://github.com/mikenakis/Bathyscaphe

Status of the project

The "Technology Readiness Level" (TRL) of the project is "5: Technology validated in lab". 

The library works, it appears to be problem-free, and it produces very good results. However, the only environment in which it is currently being put into use is the author's hobby projects, which is about as good as laboratory use. Bathyscaphe will need to receive some extensive beta testing in at least a few commercial-scale environments before it can be considered as ready for general availability.

In the mean time, Bathyscaphe is likely to undergo extensive refactoring: my understanding of certain concepts may change, and as a result the names of those concepts will inevitably have to change, which in turn means that classes and methods may be renamed. Therefore, at this early point in time there is a conundrum associated with integrating Bathyscaphe into a project: 

  • Either you pick a version and you stick to it, in which case you will not be receiving improvements as Bathyscaphe evolves,
  • Or you keep upgrading to the latest version of Bathyscaphe, but with every upgrade your code will not compile anymore and you will need to modify your code in order to make it work again.

Luckily, Bathyscaphe has a very small interface, so most changes to Bathyscaphe are likely to be internal. Even if some change affects client code, it is likely to only need small and highly localized modifications. Unluckily, the interface of Bathyscaphe also includes some annotations, and annotations tend to spread far and wide, so some changes to Bathyscaphe may require extensive modifications in client software. 

So, if you decide to try Bathyscaphe in its current state, choose wisely, and use at your own risk. 

For the time being, the author does not intend to hinder the evolution of Bathyscaphe for the purpose of maintaining backwards compatibility.

Glossary

Note: some of the glossary terms (i.e. variable / invariable, extensible / inextensible) are introduced by Bathyscaphe in order to mitigate the ambiguities caused by Java's unfortunate choice to reuse certain language keywords (i.e. 'final') to mean entirely different things in different situations.
  • assessment the result of examining a type or an instance of a type to determine whether it is immutable or not. Bathyscaphe contains a large class hierarchy of assessments. The hierarchy has a few distinct sub-hierarchies, one for type assessments, one for object assessments, one for field assessments, and one for field value assessments, but they all share a common ancestor for the purpose of constructing assessment trees, where the children of an assessment are the reasons due to which the assessment was issued. See type assessment, object assessment.
  • bathyscaphe (/ˈbæθɪskeɪf/ or /ˈbæθɪskæf/) a free-diving self-propelled deep-sea submersible with a crew cabin. Being yellow is not a strict requirement. See https://en.wikipedia.org/wiki/Bathyscaphe
  • deep immutability refers to the immutability of an entire object graph reachable from a certain object, as opposed to the immutability of only that specific object. It is among the fundamental premises of Bathyscaphe that this is the only type of immutability that matters. See superficial immutability.
  • extensible a class that may be sub-classed (extended.) Corresponds to the absence of the language keyword `final` it the class definition. See inextensible.
  • inextensible a class that may not be be sub-classed (extended.) Corresponds to the presence of the language keyword `final` in the class definition. See extensible.
  • invariable a field that cannot be mutated. Corresponds to the presence of the language keyword `final` in the field definition. Note that this is entirely without regards to the question of whether the object referenced by the field is immutable or not. See variable.
  • object assessment represents the result of examining an instance of a class (an object) to determine whether it is immutable or not. Bathyscaphe has a single assessment for immutable objects, but an entire hierarchy of assessments for all the different ways in which an object can be mutable. The information contained within mutable object assessments is used to provide useful diagnostics as to why a particular assessment was issued. See assessmenttype assessment.
  • superficial immutability refers to the immutability of a single object, without regards to the immutability of objects that it references. It is among the fundamental premises of Bathyscaphe that this type of immutability is largely inconsequential. See deep immutability.
  • type assessment represents the result of examining a class to determine whether it is immutable or not. Bathyscaphe has a single assessment to represent immutable classes, but an entire hierarchy of assessments to represent all the different ways in which a class can be mutable or provisory. The information contained in a provisory type assessment is used later to guide Bathyscaphe in examining instances of that type. The information contained both in provisory and mutable type assessments provides useful diagnostics as to why a particular assessment was issued. See assessmentobject assessment.
  • variable a field that is free to mutate. Corresponds to the absence of the language keyword 'final' in the field definition. Note that this is entirely without regards to the question of whether the object referenced by the field is immutable or not. See invariable.

Appendix: Goals of Bathyscaphe

I decided to tackle the problem of immutability and write my own assessment facility with the following goals in mind:

  • I want to be able to write framework-level code which handles application objects (thus having no prior knowledge of the classes of those objects) and can assert, whenever necessary, that these objects are immutable. For example:
    • I want to have my own hash-map which asserts that any and all keys added to it are immutable. (Same for my own hash-set.)
    • I want to have my message-passing framework assert that every single message that it is asked to deliver is immutable.
  • I want immutability assessment to be perfect, i.e. there must be no false positives or false negatives, and there must be no surprises. So, for example:
    • `List.of( 1 )` must be assessed as immutable, while
    • `List.of( new StringBuilder() )` must be assessed as mutable.
  • When assessment cannot be achieved by automatic means, (as the case is, for example, with classes that perform lazy initialization,) I want to be able to achieve it by either:
    • adding special annotations to certain fields, or
    • having the class implement a self-assessment interface so that an instance of that class can be asked whether it is immutable, or 
    • adding a manual preassessment (assessment override) for that specific class.
  • When an immutability assertion fails, meaning that an object which was intended to be immutable has actually been found to be mutable, I want the assessment to contain extensive diagnostics, explaining precisely why this is so.
  • I want the immutability assessment library which achieves all this to be attractive to programmers, by being:
    • very easy to integrate
    • very easy to use
    • very small
    • having no dependencies.

Appendix: Non-goals of Bathyscaphe

  • Dealing with untrustworthy classes.
    • Immutability can always be compromised via reflection, so trying to assess the immutability of a class which may actively circumvent the assessment is a hopeless endeavor. Therefore, all classes are to be presumed as trust-worthy.
  • Dealing with buggy classes.
    • If a class promises, either by means of annotations or the self-assessment interface, that it will behave immutably, but in fact it does not, the fault is with that class, not with the immutability assessment facility.
  • Dealing with inaccessible classes.
    • Due to security restrictions, the inner workings of certain JDK classes are inaccessible. Since every single one of those classes can receive a manual preassessment, this is not an issue.
  • Dealing with farcery.
    • If we create a subclass of a mutable class and override each mutation method to just throw an exception, do we have a mutable or immutable class in our hands? Some say it is mutable; others say it is immutable; I say it is a farce, and not worthy of consideration.
  • Performance.
    • Immutability assessment can be computationally expensive, but it is only meant to be performed through assertions, so its overhead is to be suffered only on development runs. On production runs, where assertions are supposed to be disabled, the performance penalty will be zero.
  • Static assessment.
    • While it is indeed possible in many cases to conclusively assess a class as mutable or immutable by just looking at the class, in many cases (and certainly in all interesting cases) examining the class is not enough, as the example of `List.of( 1 )` vs. `List.of( new StringBuilder() )` demonstrates. Thus, the use of Bathyscaphe as a static analysis tool is not a goal.

Appendix: A note about the so-called "immutable" collections

When Java 9 was introduced, the documentation referred to the objects returned by the new `java.util.List.of( E e1 )` method as immutable lists.  Specifically, in the Java 9 API docs we read "Returns an immutable list containing one element." Then later the Java people realized that this was inaccurate, so in JDK issue 8191517 they decided among other things to "Adjust terminology to prefer 'unmodifiable' over 'immutable'." Thus, if we look at the documentation today, (for example,  in the Java 18 API documentation) it reads "Returns an unmodifiable list containing one element."

Java 9 API docs:  https://docs.oracle.com/javase/9/docs/api/java/util/List.html

JDK issue 8191517: https://bugs.openjdk.java.net/browse/JDK-8191517

Java 18 API docs: https://docs.oracle.com/en/java/javase/18/docs/api/java.base/java/util/List.html

Dropping the word "immutable" was the right thing to do, because there is no such thing as an immutable collection, at least when type erasure is involved. That's because a collection contains elements, the immutability of which it is in no position to vouch for.  

Unfortunately, the term "unmodifiable" is also problematic for describing these collections, because the term already had a meaning before `List.of()` was introduced, and the meaning was "an unmodifiable-to-you view of my collection, which is still very mutable, and any mutations it undergoes will be visible to you." Luckily, `List.of()` does better than that; it returns a list that cannot be modified by anyone. So, I would rather call it "superficially immutable" to indicate that it falls short of achieving true immutability only in the sense that it cannot guarantee deep immutability.

No comments:

Post a Comment