Last December, I read Jeff Atwood’s write-up about SunSpider, a new Javascript benchmark created by the makers of WebKit/KHTML.

The world of Javascript is an interesting one right now; it seems like most major browsers are on the eve of a major new release, many with new (& improved JS engines). With WebKit’s porting to Windows in Safari (and eventually Konqueror, I imagine), yet another player has thrown his hat into the arena. Javascript is, as Jeff rightly says, the lingua franca of the web: everything is javascript-[based|dependent] today, and rich interfaces are now the norm, rather than a pleasant surprise.

As a web applications developer, I’ve banged by head against javascript and various and sundry browser implementations at work, and worried about performances, both now and in the future1. Standardizing on a common library (cf. jQuery) helps, although the performance or functionality of jQuery plugins don’t always match those of home-grown counterparts. For instance, Brian McAllister’s Unobtrusive Table Sort Script far outpaces the popular TableSorter script for jQuery. I ended up implementing the latter at work2, simply because of the sheer size of tables we render, and the relative slow speed of many of ours users’ browsers.

Due to time constraints, I hadn’t gotten around to performing my own benchmark until today. I don’t have any fancy (or controversial) graphs like Jeff, but here’s the data anyway, followed by the raw result text. The first table is high-level summaries listing only the sum times for groups of test. The second table lists all the tests.

Somewhat to my surprise, the much-touted improvements to Firefox 3’s javascript engine weren’t consistent across the board. In some cases, it performed worse than 2.0.0.11. The clear winner was Opera’s upcoming 9.5 release (codenamed ‘Kestrel’), followed closely by Apple’s port of the WebKit engine. With respect to the latter, it’s clear that the WebKit/KHTML engine is a good one, though I look forward to an inevitable Konqueror port, since Safari on Windows is just about the ugliest interface I’ve ever seen3.

The Results

SunSpider 0.9 benchmarks by Group
Benchmark Firefox 2.0.0.11 Firefox 3.0.0 b2 Opera 9.5 b1 Safari 3.04 Internet Explorer 7.0
Total 14032.0 11195.2 6501.0 8583.4 26676.8
3d 1909.0 1821.2 572.0 1184.4 1212.8
access 1275.0 1632.4 775.4 1668.2 1731.6
bitops 3340.6 1759.0 803.2 1465.0 1552.8
controlflow 94.0 93.4 65.6 212.4 447.0
crypto 694.4 615.6 319.0 815.8 934.4
date 2049.8 628.2 734.2 621.8 796.6
math 1281.8 1121.8 456.0 1012.4 1075.6
regexp 784.2 947.0 747.0 337.6 350.0
string 2603.2 2576.6 2028.6 1265.8 18576.0
SunSpider 0.9 benchmarks by Item
Benchmark Firefox 2.0.0.11 Firefox 3.0.0 b2 Opera 9.5 b1 Safari 3.04 Internet Explorer 7.0
Total 14032.0 11195.2 6501.0 8583.4 26676.8
3d-cube 531.4 543.6 162.8 353.0 334.6
3d-morph 1078.0 921.8 231.2 490.6 412.4
3d-raytrace 299.6 355.8 178.0 340.8 465.8
access-binary-trees 140.6 160.6 50.0 143.4 409.4
access-fannkuch 422.0 629.2 365.8 703.2 687.6
access-nbody 484.6 568.4 197.0 331.2 343.8
access-nsieve 227.8 274.2 162.6 490.4 290.8
bitops-3bit-bits-in-byte 253.2 280.8 65.6 302.8 368.8
bitops-bits-in-byte 256.4 286.0 97.0 387.2 387.2
bitops-bitwise-and 2459.4 808.0 425.0 303.0 450.2
bitops-nsieve-bits 371.6 384.2 215.6 472.0 346.6
controlflow-recursive 94.0 93.4 65.6 212.4 447.0
crypto-aes 215.6 276.0 153.0 218.6 343.6
crypto-md5 244.2 166.6 81.4 297.0 297.0
crypto-sha1 234.6 173.0 84.6 300.2 293.8
date-format-tofte 693.8 409.4 272.0 259.4 390.4
date-format-xparb 1356.0 218.8 462.2 362.4 406.2
math-cordiac 665.8 527.6 197.0 559.4 453.6
math-partial-sums 375.2 383.8 171.8 228.0 281.2
math-spectral-norm 240.8 210.4 87.2 225.0 340.8
regexp-dna 784.2 947.0 747.0 337.6 350.0
string-base64 565.8 466.4 140.8 300.0 9212.6
string-fasta 415.6 443.2 390.6 297.0 438.0
string-tagcloud 512.4 396.8 365.8 237.6 1740.6
string-unpack-code 718.8 942.0 965.6 193.8 453.2
string-validate-input 390.6 328.2 165.8 237.4 6731.6
SunSpider 0.9 Benchmarks
Performed on 15 January 2007
Ben Gunnink | //heliologue.com/

- Intel Q6600 Quad-Core
- Windows XP SP2 32-bit
- 4GB RAM

All browsers were tested with extensions and plugins disabled.

Firefox 2.0.0.11

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                 14032.0ms +/- 2.4%
--------------------------------------------

  3d:                   1909.0ms +/- 4.1%
    cube:                531.4ms +/- 7.6%
    morph:              1078.0ms +/- 3.6%
    raytrace:            299.6ms +/- 10.8%

  access:               1275.0ms +/- 2.3%
    binary-trees:        140.6ms +/- 9.7%
    fannkuch:            422.0ms +/- 0.0%
    nbody:               484.6ms +/- 2.8%
    nsieve:              227.8ms +/- 4.6%

  bitops:               3340.6ms +/- 0.9%
    3bit-bits-in-byte:   253.2ms +/- 3.5%
    bits-in-byte:        256.4ms +/- 4.2%
    bitwise-and:        2459.4ms +/- 0.7%
    nsieve-bits:         371.6ms +/- 4.4%

  controlflow:            94.0ms +/- 0.0%
    recursive:            94.0ms +/- 0.0%

  crypto:                694.4ms +/- 5.2%
    aes:                 215.6ms +/- 7.7%
    md5:                 244.2ms +/- 7.0%
    sha1:                234.6ms +/- 6.0%

  date:                 2049.8ms +/- 0.4%
    format-tofte:        693.8ms +/- 3.1%
    format-xparb:       1356.0ms +/- 1.2%

  math:                 1281.8ms +/- 10.7%
    cordic:              665.8ms +/- 17.3%
    partial-sums:        375.2ms +/- 3.6%
    spectral-norm:       240.8ms +/- 9.2%

  regexp:                784.2ms +/- 5.4%
    dna:                 784.2ms +/- 5.4%

  string:               2603.2ms +/- 7.1%
    base64:              565.8ms +/- 3.8%
    fasta:               415.6ms +/- 4.3%
    tagcloud:            512.4ms +/- 34.7%
    unpack-code:         718.8ms +/- 7.9%
    validate-input:      390.6ms +/- 8.7%
    
    
    
    
Firefox 3.0.0 beta 2

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                 11195.2ms +/- 5.9%
--------------------------------------------

  3d:                   1821.2ms +/- 5.4%
    cube:                543.6ms +/- 12.5%
    morph:               921.8ms +/- 2.2%
    raytrace:            355.8ms +/- 11.9%

  access:               1632.4ms +/- 8.6%
    binary-trees:        160.6ms +/- 34.5%
    fannkuch:            629.2ms +/- 8.5%
    nbody:               568.4ms +/- 6.2%
    nsieve:              274.2ms +/- 13.1%

  bitops:               1759.0ms +/- 6.6%
    3bit-bits-in-byte:   280.8ms +/- 12.8%
    bits-in-byte:        286.0ms +/- 8.8%
    bitwise-and:         808.0ms +/- 5.1%
    nsieve-bits:         384.2ms +/- 9.8%

  controlflow:            93.4ms +/- 27.3%
    recursive:            93.4ms +/- 27.3%

  crypto:                615.6ms +/- 3.2%
    aes:                 276.0ms +/- 6.3%
    md5:                 166.6ms +/- 6.2%
    sha1:                173.0ms +/- 5.5%

  date:                  628.2ms +/- 4.4%
    format-tofte:        409.4ms +/- 3.7%
    format-xparb:        218.8ms +/- 8.1%

  math:                 1121.8ms +/- 3.8%
    cordic:              527.6ms +/- 3.8%
    partial-sums:        383.8ms +/- 4.0%
    spectral-norm:       210.4ms +/- 6.4%

  regexp:                947.0ms +/- 57.3%
    dna:                 947.0ms +/- 57.3%

  string:               2576.6ms +/- 14.3%
    base64:              466.4ms +/- 5.7%
    fasta:               443.2ms +/- 10.0%
    tagcloud:            396.8ms +/- 10.0%
    unpack-code:         942.0ms +/- 35.9%
    validate-input:      328.2ms +/- 8.9%
    
    
    

Internet Explorer 7.0.5730.13

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                  26676.8ms +/- 0.8%
--------------------------------------------

  3d:                    1212.8ms +/- 2.4%
    cube:                 334.6ms +/- 5.3%
    morph:                412.4ms +/- 2.6%
    raytrace:             465.8ms +/- 1.8%

  access:                1731.6ms +/- 1.0%
    binary-trees:         409.4ms +/- 2.1%
    fannkuch:             687.6ms +/- 0.1%
    nbody:                343.8ms +/- 0.2%
    nsieve:               290.8ms +/- 3.6%

  bitops:                1552.8ms +/- 1.7%
    3bit-bits-in-byte:    368.8ms +/- 2.9%
    bits-in-byte:         387.2ms +/- 2.2%
    bitwise-and:          450.2ms +/- 1.9%
    nsieve-bits:          346.6ms +/- 2.7%

  controlflow:            447.0ms +/- 2.4%
    recursive:            447.0ms +/- 2.4%

  crypto:                 934.4ms +/- 0.9%
    aes:                  343.6ms +/- 0.2%
    md5:                  297.0ms +/- 0.0%
    sha1:                 293.8ms +/- 3.0%

  date:                   796.6ms +/- 2.9%
    format-tofte:         390.4ms +/- 3.5%
    format-xparb:         406.2ms +/- 3.4%

  math:                  1075.6ms +/- 0.9%
    cordic:               453.6ms +/- 0.2%
    partial-sums:         281.2ms +/- 0.2%
    spectral-norm:        340.8ms +/- 2.6%

  regexp:                 350.0ms +/- 5.0%
    dna:                  350.0ms +/- 5.0%

  string:               18576.0ms +/- 0.9%
    base64:              9212.6ms +/- 0.4%
    fasta:                438.0ms +/- 0.0%
    tagcloud:            1740.6ms +/- 5.7%
    unpack-code:          453.2ms +/- 0.1%
    validate-input:      6731.6ms +/- 1.2%
    
    
    
    
Safari 3.04 beta

============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                  8583.4ms +/- 1.7%
--------------------------------------------

  3d:                   1184.4ms +/- 4.6%
    cube:                353.0ms +/- 3.1%
    morph:               490.6ms +/- 5.2%
    raytrace:            340.8ms +/- 6.4%

  access:               1668.2ms +/- 2.5%
    binary-trees:        143.4ms +/- 6.1%
    fannkuch:            703.2ms +/- 0.1%
    nbody:               331.2ms +/- 2.5%
    nsieve:              490.4ms +/- 7.7%

  bitops:               1465.0ms +/- 2.0%
    3bit-bits-in-byte:   302.8ms +/- 5.8%
    bits-in-byte:        387.2ms +/- 2.2%
    bitwise-and:         303.0ms +/- 3.6%
    nsieve-bits:         472.0ms +/- 1.9%

  controlflow:           212.4ms +/- 5.0%
    recursive:           212.4ms +/- 5.0%

  crypto:                815.8ms +/- 3.1%
    aes:                 218.6ms +/- 10.9%
    md5:                 297.0ms +/- 0.0%
    sha1:                300.2ms +/- 3.0%

  date:                  621.8ms +/- 3.5%
    format-tofte:        259.4ms +/- 4.1%
    format-xparb:        362.4ms +/- 4.6%

  math:                 1012.4ms +/- 1.6%
    cordic:              559.4ms +/- 1.5%
    partial-sums:        228.0ms +/- 5.0%
    spectral-norm:       225.0ms +/- 5.0%

  regexp:                337.6ms +/- 3.2%
    dna:                 337.6ms +/- 3.2%

  string:               1265.8ms +/- 2.2%
    base64:              300.0ms +/- 2.8%
    fasta:               297.0ms +/- 4.7%
    tagcloud:            237.6ms +/- 6.8%
    unpack-code:         193.8ms +/- 5.4%
    validate-input:      237.4ms +/- 3.7%
    
    
    
    
Opera 9.5 beta 1 (build 9613)
    
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                  6501.0ms +/- 1.2%
--------------------------------------------

  3d:                    572.0ms +/- 2.9%
    cube:                162.8ms +/- 6.4%
    morph:               231.2ms +/- 3.7%
    raytrace:            178.0ms +/- 5.7%

  access:                775.4ms +/- 2.8%
    binary-trees:         50.0ms +/- 16.7%
    fannkuch:            365.8ms +/- 2.9%
    nbody:               197.0ms +/- 5.5%
    nsieve:              162.6ms +/- 6.6%

  bitops:                803.2ms +/- 5.6%
    3bit-bits-in-byte:    65.6ms +/- 13.2%
    bits-in-byte:         97.0ms +/- 8.6%
    bitwise-and:         425.0ms +/- 6.0%
    nsieve-bits:         215.6ms +/- 4.1%

  controlflow:            65.6ms +/- 14.2%
    recursive:            65.6ms +/- 14.2%

  crypto:                319.0ms +/- 7.0%
    aes:                 153.0ms +/- 5.9%
    md5:                  81.4ms +/- 10.8%
    sha1:                 84.6ms +/- 12.6%

  date:                  734.2ms +/- 1.9%
    format-tofte:        272.0ms +/- 4.0%
    format-xparb:        462.2ms +/- 3.7%

  math:                  456.0ms +/- 3.4%
    cordic:              197.0ms +/- 5.5%
    partial-sums:        171.8ms +/- 0.3%
    spectral-norm:        87.2ms +/- 12.0%

  regexp:                747.0ms +/- 2.2%
    dna:                 747.0ms +/- 2.2%

  string:               2028.6ms +/- 2.9%
    base64:              140.8ms +/- 10.0%
    fasta:               390.6ms +/- 6.1%
    tagcloud:            365.8ms +/- 8.0%
    unpack-code:         965.6ms +/- 1.7%
    validate-input:      165.8ms +/- 6.4%
  1. Remember that javascript is single-threaded, and will be for the foreseeable future; so, despite the huge leaps in parallel processing, javascript performance is still a matter of clock speed and renderer speed[]
  2. I did have to hack a bit at it, though, to prevent sorts breaking on empty tables, and to recognize negative numbers denoted by parentheses[]
  3. Jeff has something to say about its font rendering, too: initial post, and followup[]
§1956 · January 15, 2008 · Tags: , , , , , , ·

Leave a Reply