I recently performed some cursory Javascript benchmarks with the new version of Firefox and Safari; curious about performance, I decided to do some testing of 32-bit browsers against their 64-bit counterparts. On Windows Vista x64, the only two browsers so available to me are Internet Explorer 7 and a recent nightly build of Firefox 3. The 64-bit comes from the Mozilla x86-64 project, specifically the build from 21 March 2008. The 32-bit build is a proper nightly from the same date from the official Mozilla FTP.
My hypothesis, before performing the tests, was that the 64-bit compilation would have little or no effect on the Javascript engine performance. It’s so difficult to optimize Javascript rendering, which is inherently single-threaded, and it seems likely to only benefit from a faster CPU clock than bigger memory registers. Afterward, I felt vindicated: Internet Explorer is likely the best test, and the difference was not statistically significant. In Firefox’s case, the 64-bit build was actually significantly worse, though this could easily be due to some other factor I have not taken into account; I have assumed that the source was compiled on the same date. See below for more details.
The Data Tables
| Benchmark | 32-bit (ms) | 64-bit (ms) |
|---|---|---|
| Total | 21972.4 | 19955.4 |
| 3d: cube | 346.6 | 262.0 |
| 3d: morph | 415.2 | 311.8 |
| 3d: raytrace | 471.2 | 364.8 |
| access: binary-trees | 443.0 | 321.2 |
| access: fannkuch | 739.4 | 586.6 |
| access: nbody | 346.4 | 258.6 |
| access: nsieve | 305.8 | 284.2 |
| bitops: 3bit-bits-in-byte | 409.0 | 290.0 |
| bitops: bits-in-byte | 405.4 | 290.0 |
| bitops: bitwise-and | 446.2 | 284.0 |
| bitops: nsieve-bits | 349.4 | 290.0 |
| control: recursive | 493.0 | 349.4 |
| crypto: aes | 374.2 | 296.4 |
| crypto: md5 | 330.8 | 243.2 |
| crypto: sha1 | 346.4 | 246.6 |
| date: format-tofte | 433.6 | 315.0 |
| date: format-xparb | 440.0 | 349.6 |
| math: cordic | 465.0 | 340.0 |
| math: partial-sums | 327.6 | 228.0 |
| math: spectral-norm | 393.4 | 277.6 |
| regexp: dna | 387.0 | 302.6 |
| string: base64 | 7298.0 | 8527.2 |
| string: fasta | 464.6 | 334.2 |
| string: tagcloud | 1391.4 | 1145.0 |
| string: unpack-code | 461.8 | 346.6 |
| string: validate-input | 3688.0 | 3110.8 |
| Benchmark | 32-bit (ms) | 64-bit (ms) |
|---|---|---|
| Total | 3750.2 | 7421.4 |
| 3d: cube | 147.2 | 312.6 |
| 3d: morph | 137.8 | 230.8 |
| 3d: raytrace | 147.8 | 264.4 |
| access: binary-trees | 60.6 | 126.6 |
| access: fannkuch | 261.6 | 540.6 |
| access: nbody | 173.2 | 282.8 |
| access: nsieve | 103.4 | 185.0 |
| bitops: 3bit-bits-in-byte | 85.6 | 213.6 |
| bitops: bits-in-byte | 123.0 | 271.0 |
| bitops: bitwise-and | 136.4 | 246.2 |
| bitops: nsieve-bits | 148.6 | 310.0 |
| control: recursive | 57.0 | 140.8 |
| crypto: aes | 105.0 | 195.0 |
| crypto: md5 | 78.4 | 175.2 |
| crypto: sha1 | 88.0 | 163.6 |
| date: format-tofte | 202.2 | 406.4 |
| date: format-xparb | 129.2 | 258.2 |
| math: cordic | 186.2 | 367.4 |
| math: partial-sums | 149.2 | 257.2 |
| math: spectral-norm | 96.8 | 194.2 |
| regexp: dna | 255.6 | 652.8 |
| string: base64 | 119.2 | 220.6 |
| string: fasta | 205.2 | 388.8 |
| string: tagcloud | 152.2 | 294.8 |
| string: unpack-code | 272.8 | 489.4 |
| string: validate-input | 128.0 | 233.4 |
The Raw Data
Internet Explorer x86 7.0.6001.18000
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 21972.4ms +/- 6.2%
--------------------------------------------
3d: 1233.0ms +/- 1.1%
cube: 346.6ms +/- 4.7%
morph: 415.2ms +/- 2.7%
raytrace: 471.2ms +/- 1.9%
access: 1834.6ms +/- 2.5%
binary-trees: 443.0ms +/- 3.9%
fannkuch: 739.4ms +/- 1.5%
nbody: 346.4ms +/- 7.3%
nsieve: 305.8ms +/- 5.8%
bitops: 1610.0ms +/- 3.7%
3bit-bits-in-byte: 409.0ms +/- 2.0%
bits-in-byte: 405.4ms +/- 4.8%
bitwise-and: 446.2ms +/- 5.0%
nsieve-bits: 349.4ms +/- 6.3%
controlflow: 493.0ms +/- 2.2%
recursive: 493.0ms +/- 2.2%
crypto: 1051.4ms +/- 4.6%
aes: 374.2ms +/- 5.2%
md5: 330.8ms +/- 4.9%
sha1: 346.4ms +/- 6.2%
date: 873.6ms +/- 2.3%
format-tofte: 433.6ms +/- 2.0%
format-xparb: 440.0ms +/- 3.7%
math: 1186.0ms +/- 1.6%
cordic: 465.0ms +/- 4.6%
partial-sums: 327.6ms +/- 0.2%
spectral-norm: 393.4ms +/- 4.1%
regexp: 387.0ms +/- 4.2%
dna: 387.0ms +/- 4.2%
string: 13303.8ms +/- 10.4%
base64: 7298.0ms +/- 6.8%
fasta: 464.6ms +/- 3.5%
tagcloud: 1391.4ms +/- 4.0%
unpack-code: 461.8ms +/- 3.7%
validate-input: 3688.0ms +/- 25.4%
Internet Explorer x64
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 19955.4ms +/- 1.2%
--------------------------------------------
3d: 938.6ms +/- 2.3%
cube: 262.0ms +/- 6.2%
morph: 311.8ms +/- 6.3%
raytrace: 364.8ms +/- 2.9%
access: 1450.6ms +/- 2.8%
binary-trees: 321.2ms +/- 3.3%
fannkuch: 586.6ms +/- 3.7%
nbody: 258.6ms +/- 6.6%
nsieve: 284.2ms +/- 3.1%
bitops: 1154.0ms +/- 1.7%
3bit-bits-in-byte: 290.0ms +/- 3.7%
bits-in-byte: 290.0ms +/- 7.6%
bitwise-and: 284.0ms +/- 5.8%
nsieve-bits: 290.0ms +/- 6.0%
controlflow: 349.4ms +/- 3.0%
recursive: 349.4ms +/- 3.0%
crypto: 786.2ms +/- 2.8%
aes: 296.4ms +/- 4.6%
md5: 243.2ms +/- 7.1%
sha1: 246.6ms +/- 10.3%
date: 664.6ms +/- 6.0%
format-tofte: 315.0ms +/- 11.6%
format-xparb: 349.6ms +/- 3.1%
math: 845.6ms +/- 2.5%
cordic: 340.0ms +/- 2.5%
partial-sums: 228.0ms +/- 4.5%
spectral-norm: 277.6ms +/- 6.0%
regexp: 302.6ms +/- 3.5%
dna: 302.6ms +/- 3.5%
string: 13463.8ms +/- 1.4%
base64: 8527.2ms +/- 0.9%
fasta: 334.2ms +/- 5.2%
tagcloud: 1145.0ms +/- 2.6%
unpack-code: 346.6ms +/- 6.2%
validate-input: 3110.8ms +/- 2.4%
Firefox x86 Firefox 3.0b5pre (20080322 nightly)
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 3750.2ms +/- 1.4%
--------------------------------------------
3d: 432.8ms +/- 9.8%
cube: 147.2ms +/- 17.4%
morph: 137.8ms +/- 1.2%
raytrace: 147.8ms +/- 22.7%
access: 598.8ms +/- 2.1%
binary-trees: 60.6ms +/- 5.5%
fannkuch: 261.6ms +/- 1.7%
nbody: 173.2ms +/- 6.0%
nsieve: 103.4ms +/- 1.4%
bitops: 493.6ms +/- 5.4%
3bit-bits-in-byte: 85.6ms +/- 2.2%
bits-in-byte: 123.0ms +/- 6.4%
bitwise-and: 136.4ms +/- 11.9%
nsieve-bits: 148.6ms +/- 9.2%
controlflow: 57.0ms +/- 3.1%
recursive: 57.0ms +/- 3.1%
crypto: 271.4ms +/- 7.3%
aes: 105.0ms +/- 19.4%
md5: 78.4ms +/- 3.8%
sha1: 88.0ms +/- 2.0%
date: 331.4ms +/- 12.3%
format-tofte: 202.2ms +/- 16.2%
format-xparb: 129.2ms +/- 8.3%
math: 432.2ms +/- 3.9%
cordic: 186.2ms +/- 7.6%
partial-sums: 149.2ms +/- 7.0%
spectral-norm: 96.8ms +/- 1.4%
regexp: 255.6ms +/- 8.3%
dna: 255.6ms +/- 8.3%
string: 877.4ms +/- 4.7%
base64: 119.2ms +/- 6.0%
fasta: 205.2ms +/- 10.4%
tagcloud: 152.2ms +/- 6.2%
unpack-code: 272.8ms +/- 4.1%
validate-input: 128.0ms +/- 10.3%
Firefox x64
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total: 7421.4ms +/- 1.2%
--------------------------------------------
3d: 807.8ms +/- 2.0%
cube: 312.6ms +/- 3.5%
morph: 230.8ms +/- 7.8%
raytrace: 264.4ms +/- 6.5%
access: 1135.0ms +/- 1.9%
binary-trees: 126.6ms +/- 11.7%
fannkuch: 540.6ms +/- 2.3%
nbody: 282.8ms +/- 2.9%
nsieve: 185.0ms +/- 10.4%
bitops: 1040.8ms +/- 2.3%
3bit-bits-in-byte: 213.6ms +/- 5.8%
bits-in-byte: 271.0ms +/- 4.5%
bitwise-and: 246.2ms +/- 6.8%
nsieve-bits: 310.0ms +/- 6.1%
controlflow: 140.8ms +/- 6.6%
recursive: 140.8ms +/- 6.6%
crypto: 533.8ms +/- 4.3%
aes: 195.0ms +/- 5.8%
md5: 175.2ms +/- 11.5%
sha1: 163.6ms +/- 3.5%
date: 664.6ms +/- 4.5%
format-tofte: 406.4ms +/- 4.6%
format-xparb: 258.2ms +/- 6.3%
math: 818.8ms +/- 3.8%
cordic: 367.4ms +/- 3.2%
partial-sums: 257.2ms +/- 10.5%
spectral-norm: 194.2ms +/- 4.9%
regexp: 652.8ms +/- 2.7%
dna: 652.8ms +/- 2.7%
string: 1627.0ms +/- 3.1%
base64: 220.6ms +/- 4.7%
fasta: 388.8ms +/- 4.4%
tagcloud: 294.8ms +/- 5.2%
unpack-code: 489.4ms +/- 2.9%
validate-input: 233.4ms +/- 11.1%
Look for some agregate results (which also includ Opera) at .
> On Windows Vista x64, the only two browsers so available to me are Internet Explorer 7 and a recent nightly build of Firefox 3.
That’s simply wrong, you can find stable Firefox x64 builds at http://www.vector64.com/WindowsBuilds.html
Let me rephrase: the only two 64-bit browsers available to me were Internet Explorer 7 and Firefox; of those two, I used Internet Explorer 7 and a recent nightly build of Firefox 3.
I think that the Firefox build that you got from Makoto’s site is unoptimized, and yes, the performance is awful. I think that he just did that as a proof of concept and that building with normal compiler optimizations will come down the road.
I believe that IE7 is built with PGO which adds a big boost. FF 3.0 on Win32 will use PGO as well. I’d suggest that you give my 64-bit builds a try. I only have 2.0 builds right now. I assumed that FF 3.0 would support 64-bits natively but I’m wrong on that. I do intend to do 64-bit FF 3.0 builds but it will take me a while to get there.
Thanks for the heads up, Michael. I purposely didn’t test 2.0 builds because the JS performance is so bad, regardless of platform, but maybe I should just for illustrative purposes.
I must say, I’m a bit peeved that Mozilla hasn’t bothered to produce x64 builds themselves; I get that most plugins wouldn’t work, but that’s hardly their problem.
Makoto started work on 64-bit builds in 2003 and I spun my wheels trying to get it to work for a while. He gave me his code in 2005 and I started building them then. I thought that support would come in FF 3 but it was only NSPR support. So I need to either port my 2.0 stuff to 3.0 or ask Makoto for his stuff and then get it to work (not that easy to do).
I have a project that I haven’t really started, to get plugins to work on the 64-bit browser but I haven’t had the time for it. This already exists on Linux. It turns the plugin protocol into RPCs which are shipped to a 32-bit stub that interfaces with the 32-bit plugins. In a sense, it allows a 64-bit program to make calls into 32-bit shareable images. The author of the Linux software indicated to me that he was looking at doing a Windows port but this was early in 2007 so I assume that he’s been busy with other stuff.
Windows XP x64 edition is a very nice operating system but AMD and Microsoft dropped the ball on marketing and drivers.