Now that Java 8 is out, I was toying in my mind with the concept of a new assertion mechanism which uses lambdas. The idea is to have a central assertion method that works as follows: if assertions are enabled, a supplied method gets invoked to evaluate the assertion expression, and if it returns false, then another supplied method gets invoked to throw an exception. If assertions are not enabled, the assertion method returns without invoking the supplied merhod. This would provide more control over whether assertions are enabled or not for individual pieces of code, as well as over the type of exception thrown if the assertion fails. It would also have the nice-to-have side effect of making 100% code coverage achievable, albeit only apparently so.
Naturally, I wondered whether the performance of such a construct would be comparable to the performance of existing constructs, namely, the 'assert expression' construct and the 'if( checking && expression ) throw ...' construct. I was not hoping for equal performance, not even ballpark equal, just within the same order of magnitude.
Well, the result of the benchmark blew my mind.
Congratulations to the guys that made Java 8, because it turns out that all three constructs take roughly the same amount of time to execute!
Here is my code:
2014-07-18
Benchmarking code written in Java or C# (or any GCed, JITted, VM-based language)
Sometimes we need to measure the time it takes for various pieces of code to execute in order to determine whether a certain construct takes significantly less time to execute than another. It sounds like a pretty simple task, but anyone who has ever attempted to do it knows that simplistic approaches are highly inaccurate, and achieving any accuracy at all is not trivial.
Back in the days of C and MS-DOS things were pretty straightforward: you would read the value of the system clock, run your code, read the value of the clock again, subtract the two, and that was how much time it took to run your code. The rather coarse resolution of the system clock would skew things a bit, so one trick you would at the very least employ was to loop waiting for the value of the system clock to change, then start running your code, and stop running at another transition of the value of the system clock. Another popular hack was to run benchmarks with interrupts disabled. Yes, back in those days the entire machine was yours, so you could actually do such a thing.
Nowadays, things are far more complicated. For one thing, the entire machine tends to never be yours, so you cannot disable interrupts. Other threads will pre-empt your thread, and there is nothing you can do about it, you just have to accept some inaccuracy from it. Luckily, with modern multi-core CPUs this is not so much an issue as it used to be, but in modern VM-based languages like Java and C# we have additional and far more severe inaccuracies introduced by the garbage collection and the jitting. Luckily, their impact can be reduced.
In order to avoid inaccuracies due to jitting, we always perform one run of the code under measurement before the measurements begin. This gives the JIT compiler a chance to do its job, so it will not be getting in the way later, during the actual benchmark.
Back in the days of C and MS-DOS things were pretty straightforward: you would read the value of the system clock, run your code, read the value of the clock again, subtract the two, and that was how much time it took to run your code. The rather coarse resolution of the system clock would skew things a bit, so one trick you would at the very least employ was to loop waiting for the value of the system clock to change, then start running your code, and stop running at another transition of the value of the system clock. Another popular hack was to run benchmarks with interrupts disabled. Yes, back in those days the entire machine was yours, so you could actually do such a thing.
Nowadays, things are far more complicated. For one thing, the entire machine tends to never be yours, so you cannot disable interrupts. Other threads will pre-empt your thread, and there is nothing you can do about it, you just have to accept some inaccuracy from it. Luckily, with modern multi-core CPUs this is not so much an issue as it used to be, but in modern VM-based languages like Java and C# we have additional and far more severe inaccuracies introduced by the garbage collection and the jitting. Luckily, their impact can be reduced.
In order to avoid inaccuracies due to jitting, we always perform one run of the code under measurement before the measurements begin. This gives the JIT compiler a chance to do its job, so it will not be getting in the way later, during the actual benchmark.
2014-07-14
What do you need a debugger for?
In my many years of experience in programming I have noticed that there are some programmers who refuse to use a debugger, or try to use the debugger as little as possible, as in, only when they run out of alternative options. They tend to rely solely on the diagnostic log to troubleshoot problems in their code, so their code tends to spew thousands of lines of log entries per second, and they keep trying to divine the causes of exceptions by just looking at post-mortem stack traces.
Quite often these people do not understand what usefulness others find in debuggers. I once requested the lead developer of a certain shop (Powernet, Athens, Greece, circa 2000) to enable debugging for me on their development web server so that I can run my debugger on the web site that I was developing in that shop, and she asked me "what do you need a debugger for?" Luckily, she proceeded to fulfil my request after a couple of long seconds of me staring blankly at her.
Listen folks, if you want to be called a "programmer" and if you want to be worth the cost of the keyboard you are pounding on, the debugger needs to be your absolute first tool of choice at the slightest need for troubleshooting, not your last tool of choice, not even your second tool of choice. Companies that develop IDEs go through huge pains to provide us with nice sleek and powerful debuggers so that we can do our job better, don't you dare let their efforts go to waste.
A call stack trace in the diagnostic log of your program will tell you which function was called by which function, and that's all. This is enough in many simple cases, but when things get just slightly complicated, (and they usually do,) it is not enough. Lacking any additional information, what you end up doing is theorizing about what might have happened instead of looking and seeing what has happened.
Quite often these people do not understand what usefulness others find in debuggers. I once requested the lead developer of a certain shop (Powernet, Athens, Greece, circa 2000) to enable debugging for me on their development web server so that I can run my debugger on the web site that I was developing in that shop, and she asked me "what do you need a debugger for?" Luckily, she proceeded to fulfil my request after a couple of long seconds of me staring blankly at her.
Listen folks, if you want to be called a "programmer" and if you want to be worth the cost of the keyboard you are pounding on, the debugger needs to be your absolute first tool of choice at the slightest need for troubleshooting, not your last tool of choice, not even your second tool of choice. Companies that develop IDEs go through huge pains to provide us with nice sleek and powerful debuggers so that we can do our job better, don't you dare let their efforts go to waste.
A call stack trace in the diagnostic log of your program will tell you which function was called by which function, and that's all. This is enough in many simple cases, but when things get just slightly complicated, (and they usually do,) it is not enough. Lacking any additional information, what you end up doing is theorizing about what might have happened instead of looking and seeing what has happened.
Subscribe to:
Posts (Atom)