I'm sure that some of you all have been following ARE WE FAST YET? which has been tracking Mozilla's JavaScript performance progress against Google's V8 JavaScript engine and Apple's Nitro JavaScript engine.
The tests plotted on AWFY are Apple's SunSpider JavaScript benchmark and Google's V8 JavaScript benchmark and they're performed using the raw JS engines compiled from the latest source code from Apple, Google, and Mozilla. That means it's tracking the very leading edge of those three JavaScript engines.
JavaScript engines tracked at Are We Fast Yet.
In those charts, the yellow and black lines are two separate "just in time" compilers (JITs) that Mozilla includes in its JavaScript engine. JITs take JavaScript code and compile it so it runs a lot faster. The yellow line is Mozilla's tracing JIT, which first shipped in Firefox 3.5 and the black line is Mozilla's method JIT which will first ship in Firefox 4. For a time, they were measured separately, but as they are actually complementary technologies, they are meant to work together. That's what you see with the purple line, the method and tracing JITs working together. That's what's shipping in Firefox 4.
I'm really excited about these trendlines showing Mozilla's JS engine performance moving ever closer to the competition on the competition's benchmarks. It looks like we'll pass Apple's JS engine on Google's test soon and we're getting really close to both of those engines on Apple's test.
But those two benchmarks provided by Apple and Google are not the end all be all of JavaScript performance and so Mozilla has recently released its own benchmark called Kraken.
The Kraken JavaScript engine tests are designed to test more real-world use cases than Apple's or Google's tests currently do. Here's how Mozilla's Rob Sayre put it:
More than Sunspider, V8, and Dromaeo, Kraken focuses on realistic workloads and forward-looking applications. We believe that the benchmarks used in Kraken are better in terms of reflecting realistic workloads for pushing the edge of browser performance forward. These are the things that people are saying are too slow to do with open web technologies today, and we want to have benchmarks that reflect progress against making these near-future apps universally available.
Here are Kraken test runs I charted for the latest Internet Explorer, Chrome, and Firefox builds.
Internet Explorer 9, Google Chrome 7, and Firefox 4 preview releases compared using the Kraken JavaScript benchmark.
The test suite is broken up into several categories. Each of those categories is made up of multiple JavaScript tests executed multiple times. The visualization gets pretty complicated if I try to show all the sub-tests, but it wouldn't tell the whole story if I just report the top line scores. So I'm going with the middle ground. In this chart, you can see the different categories of test and how the three browsers compare in those categories and how that all stacks up to a total score. (You can click through the links under the chart to see the raw test data if you're so inclined.)
Firefox is ahead of both IE and Chrome overall, and in each of the categories except the artificial intelligence (AI) category where Chrome has a very slight advantage. Let's see if we can't take that crown before Firefox 4 ships ;-)
Where Firefox really shines is on the image manipulation tests, especially the gaussian blur test. That link is actually a super-cool thing about the Kraken benchmarks. You can drill down to the actual tests and see exactly what they're measuring. That way you can be sure that the tests are real and not just synthetic measures made to make a particular browser look good.
In the case of a test like gaussian blur, imagine if Photoshop was to come to the Web, to be written as a JavaScript app rather than a native code app for the desktop. Well, common filters like gaussian blur would need to perform just as well when written in JavaScript as they do in native desktop code. Right now, they don't. Even in Firefox, which gets the best score on the gaussian blur test, it takes seconds to complete the test where in my Photoshop desktop app, the same effect on the same image takes well under a second.
Looking over some of the other tests, you can start to imagine the need for things like super-fast JavaScript AI for driving the non-player characters in a MMORPG, or instantaneous JS beat detection for a killer collaborative online DJ'ing program.
By constructing tests like those in the Kraken benchmark we can start to measure our JavaScript performance for the real world tasks that Web developers need to be faster if we're going to see rich apps like Photoshop, World of Warcraft, or Zulu DJ migrate to the Open Web. When all of the browsers can really kill these tests, we'll be in a world where truly full-featured Web apps can start to replace their desktop counterparts. Until then, all of the browser vendors have a ways to go.
So poke around the results and see what's going on under the covers. Run some tests yourself. Compare your favorite browsers. And imagine what the Web could do -- and will do when all of the browsers get truly fast.
Full story at http://weblogs.mozillazine.org/asa/archives/2010/09/javascript_performan.html
Posts
No comments:
Post a Comment