Last week Internet Explorer released the latest Platform Preview (#7), and boasted of some new SunSpider test results to go along, showing IE9 PP7 as the fastest browser available in terms of JavaScript, according to the SunSpider results.  Just to make clear, SunSpider is a test developed by Webkit to test how well a particular browser handles JavaScript.

At about the same time, Adrian Kingsley-Hughes at ZDNet posted on some questions around some anomalies in the IE9 SunSpider results, having picked up on an accusatory post over at the Digitizor, which in turn took a fairly innocent paragraph in Mozilla engineer Rob Sayre’s post on SunSpider results for Firefox

One last issue that can crop up has to do with over-specialization for a specific test. While I was running the SunSpider tests above, I noticed that IE9 got a score that was at least 10x faster than every other browser on SunSpider’s math-cordic test. That would be an impressive result, but it doesn’t seem to hold up in the presence of minor variations. I made a few variations on the test: one with an extra "true;" statement (diff), and one with a "return;" statement (diff). You can run those two tests along with the original math-cordic.js file here.

All three tests should return approximately the same timing results, so a result like the one pictured above would indicate a problem of some sort.

After a flurry of Twitter retweets, an appearance on Slashdot, some on-the-fly blog post title editing to calm down on the “cheating” allegations, and even some backlash from within ZDNet, a number of bloggers posted their views:

• Lies, damned lies, and benchmarks- is IE9 cheating at SunSpider- Ars Technica
• Internet Explorer 9 Is Not Cheating on SunSpider Benchmark OS News
• Microsoft’s new IE9 triggers speed-test squabble CNET News

The IE blog responded:

Briefly, the IE9 JavaScript engine includes many different changes to improve the performance of real-world Web sites and applications. You can see this in action by visiting www.ietestdrive.com and trying the samples there with IE9 and other browsers. The behavior of the IE9 JavaScript engine is not a “special case optimization” for any benchmark and not a bug.

Some of the optimizations we’ve made to the JavaScript interpreter/compiler in IE9 are of a type known in the compiler world as dead code elimination. Dead code elimination optimizations look for code that has no effect on a running program, and removes the code from the program. This has a benefit of both reducing the size of the compiled program in memory and running the program faster.

However, Rob Sayre responded in turn to that update, giving another in depth analysis of our new favorite term, “dead code elimination”.  In the post (and we wonder if we, or the IE team, are the “beginners” he refers to in his blog post title   ) he turns the tables back on IE:

Curiously, the IE9 team has chosen not to eliminate more expensive operations such as multiplication or division. The set of analyzed operators do happen to be the ones that appear in the math-cordic benchmark.

…

The timing of my post relative to IE9 PP7’s release was purely coincidental, and I’m glad all browsers are getting faster at JavaScript execution. That said, I don’t think it’s possible to consider IE9’s implementation of Dead Code Elimination to be a serious general purpose optimization. It seems clear that it hasn’t been tested on anything other than SunSpider.

We’re interested to hear more about IE and SunSpider, but if Sayre’s latest post is correct, IE still has some explaining to do.