After the initial learning curve, this has all been going swimmingly. I merged my first development branch into the trunk yesterday, and this branch just so happens to dovetail nicely into the whole point of this post, which is the YUI compressor, an open-source javascript and CSS minification tool developed by Yahoo’s YUI team.

The Problem

To understand quickly why one should minify production client-side code, consider only that with the upward trend of size in javascript libraries (and the necessary files for such libraries), it’s possibly to be downloading a lot of client-side code in a typical web application, especially as its scope grows.

For a long time, I was using Dean Edward’s Packer, as was everyone, because its Base62 encoding produced the very lowest file size. However, what should have been obvious to everyone is that eval(bunch_of_stuff_goes_here) is making the browser do a lot more work, and this is a problem on dinosaurs like IE6.

To make matters worse, the nature of such encoding also meant that for servers which tried to compress outgoing content like javascript (either with zlib or gzip), the compression ratio suffered. Just look at this table that Julien Lecomte posted last August.

I said to myself, “Hey, we use a lot of this stuff, and we still support a lot of users with slow computers and slow browsers.” So, I moved our project from a Packer-based compression to a YUI-based compression, and turned on server-side GZIP compression for javascript files. The only problem was that I was storing the javascript files already minified, and simply pasted them into a large a couple of large global .js files. I had to keep a separate source directory, along with any customizations.

This got to be a pain in the ass, as you might well expect, and so when it occurred to me that I might be able to use the YUI library at build-time in our Netbeans project, I immediately sprung into action.

This was around April, and one night while attending Sungard Summit in Anaheim. I did the initial work to get our Netbeans project into a state that could use the YUI compressor at build-time, creating separate source file directories and breaking our massive javascript file into modules; I did the same with our CSS, splitting it up based on what it decorated.

There are a few tutorials about using the YUI library. Some of them involve adding the YUI library to Ant’s classpath (didn’t want to go down this route); a lot of them involve invoking the library as an external executable during the build process, which is messy.

The solution I finally settled on was yui-compressor-ant-task, a small library that allows Ant to use YUI as a build task. By adding this library and the YUI compressor library to our common libraries folder, and enabling them at build only (and not for deploying in the web archive), it makes using the compressor pretty easy.

Here’s part of our build.xml:

<!--
   * minify will concatenate all of our non-TinyMCE javascripts and stylesheets
   * then use the YUI compressor library to compress them
   -->
   <target name="minify">
       <!--${libs} is path to the downloaded jars -->
       <property
           name="yui-compressor.jar"
           location="${file.reference.yuicompressor.jar}" />
       <property
           name="yui-compressor-ant-task.jar"
           location="${file.reference.yui-compressor-ant-task.jar}" />
 
       <path id="task.classpath">
           <pathelement location="${yui-compressor.jar}" />
           <pathelement location="${yui-compressor-ant-task.jar}" />
       </path>
 
       <!-- yui-compressor task definition -->
       <taskdef
           name="yui-compressor"
           classname="net.noha.tools.ant.yuicompressor.tasks.YuiCompressorTask">
 
           <classpath refid="task.classpath" />
       </taskdef>
 
       <!-- concatenation of javascript -->
       <echo message="Building global javascript" />
       <concat destfile="${build.dir}/web/common/js/global.js" force="no">
           <!-- explicitly order js concat because ordering matters here -->
           <fileset dir="${build.dir}" includes="web/common/js/jquery.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.bgiframe.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.hoverIntent.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.core.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.autocomplete.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.datepicker.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.tabs.js" />
           <fileset dir="${build.dir}" includes="web/common/js/tablesort.js" />
           <fileset dir="${build.dir}" includes="web/common/js/customsort.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.blockUI.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.form.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.ifixpng.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.superfish.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.cluetip.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.scrollTo.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.jqModal.js" />
           <fileset dir="${build.dir}" includes="web/common/js/validation.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.timeentry.js" />
       </concat>
 
       <!-- concatenation of cascading stylesheets -->
       <echo message="Building global cascading stylesheets" />
       <concat destfile="${build.dir}/web/common/css/global.css" force="no">
           <fileset dir="${build.dir}" includes="web/common/css/base.css" />
           <fileset dir="${build.dir}" includes="web/common/css/superfish.css" />
           <fileset dir="${build.dir}" includes="web/common/css/announcements.css" />
           <fileset dir="${build.dir}" includes="web/common/css/myvt.css" />
           <fileset dir="${build.dir}" includes="web/common/css/forms.css" />
           <fileset dir="${build.dir}" includes="web/common/css/cluetip.css" />   
           <fileset dir="${build.dir}" includes="web/common/css/tables.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.tabs.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.datepicker.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.autocomplete.css" />
           <fileset dir="${build.dir}" includes="web/common/css/linkspan.css" />
           <fileset dir="${build.dir}" includes="web/common/css/stepMenu.css" />
           <fileset dir="${build.dir}" includes="web/common/css/print.css" />
           <fileset dir="${build.dir}" includes="web/common/css/youHaveMessages.css" />
       </concat>
 
       <!-- invoke compressor -->
       <yui-compressor warn="false" charset="UTF-8" fromdir="${build.dir}" todir="${build.dir}">
           <include name="web/common/js/global.js" />
           <include name="web/common/css/global.css" />
       </yui-compressor>
 
   </target>
 
   <!--
   * purge-src takes our compressed files, moves them to the base /common dir
   * and deletes the source js and css dirs from the build dir
   -->
   <target name="purge-src" depends="minify">
       <echo message="Purging javascript and stylesheet sources" />
 
       <move file="${build.dir}/web/common/js/global-min.js" tofile="${build.dir}/web/common/global.js"/>
       <move file="${build.dir}/web/common/css/global-min.css" tofile="${build.dir}/web/common/global.css"/>
 
       <delete dir="${build.dir}/web/common/js" />
       <delete dir="${build.dir}/web/common/css" />
 
   </target>

What you see there is essentially four steps.

1. Concatenate all the constituent source files into a global.js and a global.css
2. Compress both of these files, which creates global-min.js and global-min.css (default behavior)
3. Move these files out of the source directories and into the root of the common web directory as global.js and global.css
4. Delete the source directories in our build folder so they don’t get deployed with the web archive

Because certain browsers (IE) break without explicit ordering, we unfortunately can’t just use “*.js” and “*.css” in our concatenation step, but having to explicitly list our components in the build file certainly isn’t the end of the world. The nice thing is that the Ant task will even print out handy statistics on just how much you’ve been able to compress the files down. In our case, we have about 441.8KB of common Javascript and CSS in our source code that, by the time it gets sent to the user, has been minified and/or gzipped to about 89KB.

I personally think the friendly rivalry between the open-source Webkit engine (which powers Safari, among other things) and Mozilla (the Gecko engine, actually) is one of the best things to happen to browser development in years. The constant one-upsmanship can only lead to better browsers. Well, Internet Explorer will constantly be the limiting factor, but we can dream, can’t we?

The latest thing to hit browser source repos is javascript engine improvement based on something called “trace trees”: essentially, javascript gets translated into native bytecode. The Webkit engine made the announcement a few months ago, with code codenamed “Squirrelfish,” promising massive improvements. That article’s also got a pretty good writeup.

Open Source being what it is, it was only a matter of time before Mozilla announced their own version of a trace-tree-based javascript engine. John Resig has a good writeup, as done Brendan Eich.

So, disregarding for a moment all talk of trace trees and JIT, what the hell does this all mean? I am entirely in agreement with Jeff Atwood about the importance of javascript in the contemporary web. Whether you’re using rich sites like Netflix or GMail, or you’re working with Flash files, or you’re getting into the RIA arena with Flex or similar technologies, you’re essentially relying on javascript and javascript engines to get anything done. Good HTML specs get the browsing experience much of the way there, but a truly compelling web technology relies on client-side scripting. Even in the last 3 or 4 years, javascript has gone from superfluous UI tricks to the heart and soul of the user’s end of a web experience. It’s a bit unfortunate, really, since javascript (properly “ECMAScript”) is an old technology that’s single-threaded and doesn’t scale very well. That browser-makers have managed to squeeze this much performance out of it is a testament to geek creativity.

There’s been a lot of reason to switch away from Internet Explorer in the last few years. Microsoft made big steps with IE7 to improvement performance, rendering quality, and security, but Internet Explorer is still the bastard child of the browser world, dog-ass slow compared to its brethren. If the difference between user experience in the javascript-heavy web becomes so great that a Webkit browser or a Gecko browser or Opera provides an entirely different level of satisfaction, then maybe we’ll finally see a real shift in browser usage.

Mozilla, particularly, is posed to reap some real reward. These JIT trace tree improvements will start showing up in 3.1 (I think), and much more so with the new Tamarin engine schedule for Firefox 4. The only viable Webkit browser in Windows, meanwhile, is Safari, which looks and feels alien in Windows, as Apple has a strange insistence upon keep the OSX look, regardless of platform. I don’t know offhand the market slice that Safari for Windows occupies, but it must be infinitesimally small, even compared to Safari on Mac. Firefox, then, is the only truly compelling browser on the Windows platform to take advantage of this right now.

My hypothesis, before performing the tests, was that the 64-bit compilation would have little or no effect on the Javascript engine performance. It’s so difficult to optimize Javascript rendering, which is inherently single-threaded, and it seems likely to only benefit from a faster CPU clock than bigger memory registers. Afterward, I felt vindicated: Internet Explorer is likely the best test, and the difference was not statistically significant. In Firefox’s case, the 64-bit build was actually significantly worse, though this could easily be due to some other factor I have not taken into account; I have assumed that the source was compiled on the same date. See below for more details.

The Data Tables

The Raw Data

Internet Explorer x86 7.0.6001.18000
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                  21972.4ms +/- 6.2%
--------------------------------------------

  3d:                    1233.0ms +/- 1.1%
    cube:                 346.6ms +/- 4.7%
    morph:                415.2ms +/- 2.7%
    raytrace:             471.2ms +/- 1.9%

  access:                1834.6ms +/- 2.5%
    binary-trees:         443.0ms +/- 3.9%
    fannkuch:             739.4ms +/- 1.5%
    nbody:                346.4ms +/- 7.3%
    nsieve:               305.8ms +/- 5.8%

  bitops:                1610.0ms +/- 3.7%
    3bit-bits-in-byte:    409.0ms +/- 2.0%
    bits-in-byte:         405.4ms +/- 4.8%
    bitwise-and:          446.2ms +/- 5.0%
    nsieve-bits:          349.4ms +/- 6.3%

  controlflow:            493.0ms +/- 2.2%
    recursive:            493.0ms +/- 2.2%

  crypto:                1051.4ms +/- 4.6%
    aes:                  374.2ms +/- 5.2%
    md5:                  330.8ms +/- 4.9%
    sha1:                 346.4ms +/- 6.2%

  date:                   873.6ms +/- 2.3%
    format-tofte:         433.6ms +/- 2.0%
    format-xparb:         440.0ms +/- 3.7%

  math:                  1186.0ms +/- 1.6%
    cordic:               465.0ms +/- 4.6%
    partial-sums:         327.6ms +/- 0.2%
    spectral-norm:        393.4ms +/- 4.1%

  regexp:                 387.0ms +/- 4.2%
    dna:                  387.0ms +/- 4.2%

  string:               13303.8ms +/- 10.4%
    base64:              7298.0ms +/- 6.8%
    fasta:                464.6ms +/- 3.5%
    tagcloud:            1391.4ms +/- 4.0%
    unpack-code:          461.8ms +/- 3.7%
    validate-input:      3688.0ms +/- 25.4%

Internet Explorer x64
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                  19955.4ms +/- 1.2%
--------------------------------------------

  3d:                     938.6ms +/- 2.3%
    cube:                 262.0ms +/- 6.2%
    morph:                311.8ms +/- 6.3%
    raytrace:             364.8ms +/- 2.9%

  access:                1450.6ms +/- 2.8%
    binary-trees:         321.2ms +/- 3.3%
    fannkuch:             586.6ms +/- 3.7%
    nbody:                258.6ms +/- 6.6%
    nsieve:               284.2ms +/- 3.1%

  bitops:                1154.0ms +/- 1.7%
    3bit-bits-in-byte:    290.0ms +/- 3.7%
    bits-in-byte:         290.0ms +/- 7.6%
    bitwise-and:          284.0ms +/- 5.8%
    nsieve-bits:          290.0ms +/- 6.0%

  controlflow:            349.4ms +/- 3.0%
    recursive:            349.4ms +/- 3.0%

  crypto:                 786.2ms +/- 2.8%
    aes:                  296.4ms +/- 4.6%
    md5:                  243.2ms +/- 7.1%
    sha1:                 246.6ms +/- 10.3%

  date:                   664.6ms +/- 6.0%
    format-tofte:         315.0ms +/- 11.6%
    format-xparb:         349.6ms +/- 3.1%

  math:                   845.6ms +/- 2.5%
    cordic:               340.0ms +/- 2.5%
    partial-sums:         228.0ms +/- 4.5%
    spectral-norm:        277.6ms +/- 6.0%

  regexp:                 302.6ms +/- 3.5%
    dna:                  302.6ms +/- 3.5%

  string:               13463.8ms +/- 1.4%
    base64:              8527.2ms +/- 0.9%
    fasta:                334.2ms +/- 5.2%
    tagcloud:            1145.0ms +/- 2.6%
    unpack-code:          346.6ms +/- 6.2%
    validate-input:      3110.8ms +/- 2.4%

Firefox x86 Firefox 3.0b5pre (20080322 nightly)
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                 3750.2ms +/- 1.4%
--------------------------------------------

  3d:                   432.8ms +/- 9.8%
    cube:               147.2ms +/- 17.4%
    morph:              137.8ms +/- 1.2%
    raytrace:           147.8ms +/- 22.7%

  access:               598.8ms +/- 2.1%
    binary-trees:        60.6ms +/- 5.5%
    fannkuch:           261.6ms +/- 1.7%
    nbody:              173.2ms +/- 6.0%
    nsieve:             103.4ms +/- 1.4%

  bitops:               493.6ms +/- 5.4%
    3bit-bits-in-byte:   85.6ms +/- 2.2%
    bits-in-byte:       123.0ms +/- 6.4%
    bitwise-and:        136.4ms +/- 11.9%
    nsieve-bits:        148.6ms +/- 9.2%

  controlflow:           57.0ms +/- 3.1%
    recursive:           57.0ms +/- 3.1%

  crypto:               271.4ms +/- 7.3%
    aes:                105.0ms +/- 19.4%
    md5:                 78.4ms +/- 3.8%
    sha1:                88.0ms +/- 2.0%

  date:                 331.4ms +/- 12.3%
    format-tofte:       202.2ms +/- 16.2%
    format-xparb:       129.2ms +/- 8.3%

  math:                 432.2ms +/- 3.9%
    cordic:             186.2ms +/- 7.6%
    partial-sums:       149.2ms +/- 7.0%
    spectral-norm:       96.8ms +/- 1.4%

  regexp:               255.6ms +/- 8.3%
    dna:                255.6ms +/- 8.3%

  string:               877.4ms +/- 4.7%
    base64:             119.2ms +/- 6.0%
    fasta:              205.2ms +/- 10.4%
    tagcloud:           152.2ms +/- 6.2%
    unpack-code:        272.8ms +/- 4.1%
    validate-input:     128.0ms +/- 10.3%

Firefox x64
============================================
RESULTS (means and 95% confidence intervals)
--------------------------------------------
Total:                  7421.4ms +/- 1.2%
--------------------------------------------

  3d:                    807.8ms +/- 2.0%
    cube:                312.6ms +/- 3.5%
    morph:               230.8ms +/- 7.8%
    raytrace:            264.4ms +/- 6.5%

  access:               1135.0ms +/- 1.9%
    binary-trees:        126.6ms +/- 11.7%
    fannkuch:            540.6ms +/- 2.3%
    nbody:               282.8ms +/- 2.9%
    nsieve:              185.0ms +/- 10.4%

  bitops:               1040.8ms +/- 2.3%
    3bit-bits-in-byte:   213.6ms +/- 5.8%
    bits-in-byte:        271.0ms +/- 4.5%
    bitwise-and:         246.2ms +/- 6.8%
    nsieve-bits:         310.0ms +/- 6.1%

  controlflow:           140.8ms +/- 6.6%
    recursive:           140.8ms +/- 6.6%

  crypto:                533.8ms +/- 4.3%
    aes:                 195.0ms +/- 5.8%
    md5:                 175.2ms +/- 11.5%
    sha1:                163.6ms +/- 3.5%

  date:                  664.6ms +/- 4.5%
    format-tofte:        406.4ms +/- 4.6%
    format-xparb:        258.2ms +/- 6.3%

  math:                  818.8ms +/- 3.8%
    cordic:              367.4ms +/- 3.2%
    partial-sums:        257.2ms +/- 10.5%
    spectral-norm:       194.2ms +/- 4.9%

  regexp:                652.8ms +/- 2.7%
    dna:                 652.8ms +/- 2.7%

  string:               1627.0ms +/- 3.1%
    base64:              220.6ms +/- 4.7%
    fasta:               388.8ms +/- 4.4%
    tagcloud:            294.8ms +/- 5.2%
    unpack-code:         489.4ms +/- 2.9%
    validate-input:      233.4ms +/- 11.1%