Build configurations

Abstract

The popular practice of having only two different kinds of builds (Debug and Release) is shown to be inadequate. Four different kinds of builds are proposed instead, allowing potentially better performance when running tests on a build server, better performance of the final shipped product, and more thorough error checking during development.

(Useful pre-reading: About these papers)

The Issue

In software development we often want our source code to have different characteristics under different circumstances; for example:

• Optimizations:
  
• While developing we usually do not want them, because they interfere with debugging.
• On the final shipped product we want them, because they make it run faster.
• Preconditions, assertions, and other kinds of runtime checks:
• While developing we want them, because they help us catch bugs.
• On the final shipped product we do not want them, because they slow it down.

The History

In C and C++, different behavior has historically been achieved by means of compiler options controlling optimization, and preprocessor macros controlling conditional compilation. The standard stipulates an NDEBUG macro which, if defined, causes assertions to compile to nothing. This means that software systems written in C and C++ generally have two builds: a Debug build, for use while debugging, and a Release build, for shipping or deploying to production.

When Java came along, it was decided that a single build should be good for everyone: conditional compilation was abolished, [see footnote 1,] all optimization-related choices were delegated to the Just-In-Time compiler (JITter,) assertions were made to always compile into the binaries, and the `--enableassertions` switch was added to the virtual machine for controlling during runtime, rather than during compilation, whether assertions should execute or not. This essentially gives Java developers the ability to choose between a debug run or a release run, as opposed to a debug build or a release build.

C# has brought back a compiler option for controlling optimization, and a simplified version of the preprocessor macros (called "define constants" in C#) along with special compiler support for conditional compilation, by means of the `Conditional` attribute. The build system has explicit support for different kinds of builds, called Build Configurations. Two configurations are predefined: Debug and Release. The build system offers great flexibility in defining additional build configurations, but C# developers rarely bother with that.

The Problem

The vast majority of dotnet projects use only the two predefined build configurations: Debug and Release. (Many projects actually use only Debug, but let us pretend we never heard of them.) Since only two options are available, all different needs and usage scenarios are being shoe-horned to fit into one of those two options. For example:

• There is only one build of a library that can be published, namely the Release build, which means that this build is used not only in production scenarios, but also in development scenarios, where software is being developed that is making use of a published library.
• There is only one build that can be tested, namely the Debug build, which means that this build is used not only for running tests and debugging on a developer's computer, but also for running tests on build servers.

This is problematic because:

• Build servers run unoptimized tests, exercising unoptimized production code. If the tests are long-running and computationally expensive, this is adding insult to injury by needlessly making the tests run even slower. The debug configuration is unoptimized because optimizations tend to interfere with debugging, but on a build server virtually nothing ever gets debugged, so there is never a need to be running unoptimized code.
• Release builds on production environments are wasting time needlessly executing preconditions. This is happening because in the vast majority of dotnet library projects, preconditions are implemented so as to always execute, even on release builds. This is in turn being done because when developing software that is making use of a library, the software may contain bugs that may cause it to use the library in invalid ways, so we want the library to be able to detect such invalid usage. However, in an actual production scenario, the software using the library has already been tested and is known to work correctly, so there is no need for the library to be executing preconditions. (And even if the software did happen to make invalid use of the library on production, it makes very little difference whether the inevitable resulting catastrophic failure would be signaled by a precondition failure or by some index out of range exception further down.)
• Programmers often refrain from adding certain preconditions, if they suspect them to be even slightly expensive, in light of the fact that these preconditions will be executing even on release builds. The proverbial example to illustrate this scenario is the binary search function, which should, in principle, be enforcing the precondition that the array to search must be sorted. Yes, this means guarding a O(log₂(N)) operation with a O(N) operation. This is not a problem in development scenarios, because we test with small amounts of data anyway, but if preconditions are enabled in production scenarios, it is terrible. That is why the vast majority of dotnet programmers would never add this precondition to a binary search function, even though all binary search functions absolutely need it.

The Solution

From the description of the problem it becomes evident that preconditions must be controlled separately from assertions, and both of those must be controlled separately from optimizations. Therefore, four different build configurations are needed:

1. Debug
   Everyone is more or less already familiar with this. It is meant for use by a developer when testing and debugging software on their local computer. Assertions are enabled, preconditions are enabled, and optimizations are disabled, because they interfere with debugging.
2. Optimized
   This is identical to Debug except that optimizations are enabled. It is meant to run on the build server, where we do not usually debug, so there is no reason to be running unoptimized software. Note that this configuration is only useful for projects that suffer from long-running, computationally expensive tests; projects that do testing right, with very short and lightweight tests, are likely to see a performance degradation from this configuration, due to the additional JITting overhead. Also see footnote 2.
3. Develop
   This is identical to what is commonly understood as the Release configuration, where optimizations are enabled and assertions are disabled, but preconditions are enabled. It is only applicable to a library, (not an application,) and it is meant to only be used when developing software that makes use of that library. It is not meant to be shipped to production, because preconditions should not be executing on production.
4. Release
   This is similar to the Develop configuration, except that preconditions are also disabled, for maximum performance. It is the configuration which is meant for shipping to production.

Here is the feature matrix:

		Build Configurations
		Debug	Optimized	Develop	Release
Features	Optimizations disabled	✅	⬜	⬜	⬜
	Assertions enabled	✅	✅	⬜	⬜
	Overflow checking	✅	✅	⬜	⬜
	Preconditions enabled	✅	✅	✅	⬜
	Code analysis	✅	✅	⬜	⬜

Here is an excerpt from a .csproj file implementing the above matrix:

<Choose>
  <When Condition="'$(Configuration)'=='Debug'">
    <PropertyGroup>
      <Optimize>False</Optimize>
      <DefineConstants>$(DefineConstants);PRECONDITIONS;ASSERTIONS</DefineConstants>
      <CheckForOverflowUnderflow>True</CheckForOverflowUnderflow>
      <EnableNETAnalyzers>True</EnableNETAnalyzers>
      <DebugType>Full</DebugType>
    </PropertyGroup>
  </When>
  <When Condition="'$(Configuration)'=='Optimized'">
    <PropertyGroup>
      <Optimize>True</Optimize>
      <DefineConstants>$(DefineConstants);PRECONDITIONS;ASSERTIONS</DefineConstants>
      <CheckForOverflowUnderflow>True</CheckForOverflowUnderflow>
      <EnableNETAnalyzers>True</EnableNETAnalyzers>
      <DebugType>Full</DebugType>
      <OutputPath>bin\$(Platform)\$(Configuration)\</OutputPath>
    </PropertyGroup>
  </When>
  <When Condition="'$(Configuration)'=='Develop'">
    <PropertyGroup>
      <Optimize>True</Optimize>
      <DefineConstants>$(DefineConstants);PRECONDITIONS</DefineConstants>
      <CheckForOverflowUnderflow>False</CheckForOverflowUnderflow>
      <EnableNETAnalyzers>False</EnableNETAnalyzers>
      <DebugType>Portable</DebugType>
      <OutputPath>bin\$(Platform)\$(Configuration)\</OutputPath>
    </PropertyGroup>
  </When>
  <When Condition="'$(Configuration)'=='Release'">
    <PropertyGroup>
      <Optimize>True</Optimize>
      <DefineConstants>$(DefineConstants)</DefineConstants>
      <CheckForOverflowUnderflow>False</CheckForOverflowUnderflow>
      <EnableNETAnalyzers>False</EnableNETAnalyzers>
      <DebugType>Portable</DebugType>
      <Deterministic>True</Deterministic>
      <DeterministicSourcePaths>True</DeterministicSourcePaths>
    </PropertyGroup>
  </When>
  <Otherwise>
    ...
  </Otherwise>
</Choose>

If we follow this build configuration schema, then each time we publish a library we must generate two packages: the 'Develop' package, and the 'Release' package.  

• The 'Develop' package is to be referenced by software under development.
• The 'Release' package is to be referenced by software that is being shipped to production.

For any build configuration of a certain module, (either an application or a library,) the build configuration of the libraries it uses can be determined using the following table:

Build configuration of module using library	Build configuration of library
Debug	Develop
Optimized	Develop
Develop	Develop
Release	Release

Note that the Develop configuration of a module could, in theory, make use of the better-performing Release configuration of a library, instead of the Develop configuration; however, that can only work if the module does not expose the library, or if there is no other module in the solution that uses the Develop configuration of the library. Otherwise, there are going to be type mismatch errors, where code built to make use of the develop configuration of a library is given to work with the release configuration of that library, and vice versa.

Here is an excerpt of a .csproj file implementing the above table:

<PropertyGroup>
  <PackagesConfiguration Condition="'$(Configuration)'=='Debug'"    >Develop</PackagesConfiguration>
  <PackagesConfiguration Condition="'$(Configuration)'=='Optimized'">Develop</PackagesConfiguration>
  <PackagesConfiguration Condition="'$(Configuration)'=='Develop'"  >Develop</PackagesConfiguration>
  <PackagesConfiguration Condition="'$(Configuration)'=='Release'"  >Release</PackagesConfiguration>
</PropertyGroup>

Conclusions

• An 'Optimized' build configuration has been proposed, for use when running tests on build servers. It can cut in half the running time of slow, computationally expensive tests.
• A 'Develop' build configuration has been proposed for libraries, intended for use during development of software using the libraries, but not for shipping to production. It has preconditions enabled, in order to catch bugs in the modules using the libraries.
• A 'Release' build configuration has been proposed, intended for shipping to production. It improves the performance of the final shipped product by not executing preconditions.
• The adoption of this build configuration schema means that preconditions do not incur a performance penalty on production anymore, so programmers can apply preconditions more liberally, leading to more robust software.
• When a library is published, two packages should be generated: the 'Develop' package, for developing software that uses the library, and the 'Release' package, for shipping to production.

 

---

 

Footnotes

Footnote 1: The creators of Java made it so that the generation of code within an `if()` statement controlled by a compile-time constant is suppressed if that constant evaluates to `false`, but they intentionally deprived developers from the ability to specify the value of a compile-time constant via the command-line of the compiler. They defended this choice by saying that there is inherent merit in being able to guarantee that in Java every compilation unit has one and only one set of semantics. The usefulness of this merit is debatable. It can be argued that this is simply Java treating developers the same way that Apple has been treating users: as idiots.

Footnote 2: In C# most optimizations are performed by the Just-In-Time compiler (JITter), and people say that the optimizations performed by the language compiler do not make much of a difference. However, my experiments have shown otherwise: computation-intensive code tends to run twice as fast when optimizations are enabled than when not, and this difference can be observed on a build server, so it is unaffected by any optimization choices that the JITter might make due to a debugger being attached or not. I suspect that this is happening because the language compiler saves the "optimize" flag in the binary, and the JITter subsequently observes this flag.

 

---

 

Cover image generated by ChatGPT, and then retouched by michael.gr. The prompt used was: "Please give me an image conveying the concept of highly complex and highly technical software development. Make it in landscape format, of photographic quality, with warm colors" and then "Please make the programmer look more senior".