Benchmarking Java 8 lambdas

Now that Java 8 is out, I was toying in my mind with the concept of a new assertion mechanism which uses lambdas. The idea is to have a central assertion method that works as follows: if assertions are enabled, a supplied method gets invoked to evaluate the assertion expression, and if it returns false, then another supplied method gets invoked to throw an exception. If assertions are not enabled, the assertion method returns without invoking the supplied merhod. This would provide more control over whether assertions are enabled or not for individual pieces of code, as well as over the type of exception thrown if the assertion fails. It would also have the nice-to-have side effect of making 100% code coverage achievable, albeit only apparently so.

Naturally, I wondered whether the performance of such a construct would be comparable to the performance of existing constructs, namely, the 'assert expression' construct and the 'if( checking && expression ) throw ...' construct. I was not hoping for equal performance, not even ballpark equal, just within the same order of magnitude.

Well, the result of the benchmark blew my mind.

Congratulations to the guys that made Java 8, because it turns out that all three constructs take roughly the same amount of time to execute!

Here is my code:

package saganaki;

public class TestProgram
{
    public static void main( String[] arguments )
    {
        Benchmark benchmark = new Benchmark( 100 );
        for( int i = 0;  i < 3;  i++ )
        {
            run( benchmark, true );
            run( benchmark, false );
            System.out.println();
        }
    }

    interface Checker
    {
        boolean check();
    }

    private static boolean assertionsEnabled = true;

    private static void run( Benchmark benchmark, boolean enableAssertions )
    {
        TestProgram.class.getClassLoader().setClassAssertionStatus( TestProgram.class.getName(), enableAssertions );
        assertionsEnabled = enableAssertions;
        String prefix = "assertions " + (enableAssertions? " enabled" : "disabled");
        benchmark.runAndPrint( prefix + ": if-statement", new Runnable()
        {
            @Override
            public void run()
            {
                if( assertionsEnabled && System.out == null )
                    throw new IllegalArgumentException();
            }
        } );
        benchmark.runAndPrint( prefix + ": assert      ", new Runnable()
        {
            @Override
            public void run()
            {
                assert System.out != null;
            }
        } );
        benchmark.runAndPrint( prefix + ": assertTrue()", new Runnable()
        {
            @Override
            public void run()
            {
                assertTrue( () -> System.out != null, () -> { throw new IllegalArgumentException(); } );
            }
        } );
    }

    static void assertTrue( Checker checker, Runnable thrower )
    {
        if( !assertionsEnabled )
            return;
        if( checker.check() )
            return;
        thrower.run();
    }
} 

And here is the output:

assertions  enabled: if-statement 28237.0 iterations per millisecond
assertions  enabled: assert       29037.9 iterations per millisecond
assertions  enabled: assertTrue() 24593.2 iterations per millisecond
assertions disabled: if-statement 25118.5 iterations per millisecond
assertions disabled: assert       25912.2 iterations per millisecond
assertions disabled: assertTrue() 24825.6 iterations per millisecond

assertions  enabled: if-statement 25835.9 iterations per millisecond
assertions  enabled: assert       25127.6 iterations per millisecond
assertions  enabled: assertTrue() 25572.9 iterations per millisecond
assertions disabled: if-statement 25469.6 iterations per millisecond
assertions disabled: assert       25448.3 iterations per millisecond
assertions disabled: assertTrue() 25415.7 iterations per millisecond

assertions  enabled: if-statement 25838.6 iterations per millisecond
assertions  enabled: assert       25158.9 iterations per millisecond
assertions  enabled: assertTrue() 25541.7 iterations per millisecond
assertions disabled: if-statement 25373.6 iterations per millisecond
assertions disabled: assert       25402.5 iterations per millisecond
assertions disabled: assertTrue() 25370.9 iterations per millisecond

The first run shows quite different results from the next two runs, so it is best disregarded.

For the Benchmark class, see my previous post: michael.gr: Benchmarking code written in Java or C# (or any GCed, JITted, VM-based language)