Last year, I moved our small programming department from using JDeveloper and editing shared files directly on a network drive1 to using Netbeans 6.x and a proper version control system (Subversion).
After the initial learning curve, this has all been going swimmingly. I merged my first development branch into the trunk yesterday, and this branch just so happens to dovetail nicely into the whole point of this post, which is the YUI compressor, an open-source javascript and CSS minification tool developed by Yahoo’s YUI team.
The Problem
To understand quickly why one should minify production client-side code, consider only that with the upward trend of size in javascript libraries (and the necessary files for such libraries), it’s possibly to be downloading a lot of client-side code in a typical web application, especially as its scope grows.
For a long time, I was using Dean Edward’s Packer, as was everyone, because its Base62 encoding produced the very lowest file size. However, what should have been obvious to everyone is that eval(bunch_of_stuff_goes_here) is making the browser do a lot more work, and this is a problem on dinosaurs like IE62.
To make matters worse, the nature of such encoding also meant that for servers which tried to compress outgoing content like javascript (either with zlib or gzip), the compression ratio suffered. Just look at this table that Julien Lecomte posted last August.
| File | File size (bytes) | Gzipped file size (bytes) |
|---|---|---|
| Original jQuery libraries | 62,885 | 19,758 |
| jQuery minified with JSMin | 36,391 | 11,541 |
| jQuery minified with Packer | 21,557 | 11,119 |
| jQuery minified with the YUI compressor | 31,822 | 10,818 |
I said to myself, “Hey, we use a lot of this stuff, and we still support a lot of users with slow computers and slow browsers.” So, I moved our project from a Packer-based compression to a YUI-based compression, and turned on server-side GZIP compression for javascript files. The only problem was that I was storing the javascript files already minified, and simply pasted them into a large a couple of large global .js files. I had to keep a separate source directory, along with any customizations.
This got to be a pain in the ass, as you might well expect, and so when it occurred to me that I might be able to use the YUI library at build-time in our Netbeans project, I immediately sprung into action.
This was around April, and one night while attending Sungard Summit in Anaheim. I did the initial work to get our Netbeans project into a state that could use the YUI compressor at build-time, creating separate source file directories and breaking our massive javascript file into modules; I did the same with our CSS, splitting it up based on what it decorated.
There are a few tutorials about using the YUI library. Some of them involve adding the YUI library to Ant’s classpath (didn’t want to go down this route); a lot of them involve invoking the library as an external executable during the build process, which is messy.
The solution I finally settled on was yui-compressor-ant-task, a small library that allows Ant to use YUI as a build task. By adding this library and the YUI compressor library to our common libraries folder, and enabling them at build only (and not for deploying in the web archive), it makes using the compressor pretty easy.
Here’s part of our build.xml:
<!-- * minify will concatenate all of our non-TinyMCE javascripts and stylesheets * then use the YUI compressor library to compress them --> <target name="minify"> <!--${libs} is path to the downloaded jars --> <property name="yui-compressor.jar" location="${file.reference.yuicompressor.jar}" /> <property name="yui-compressor-ant-task.jar" location="${file.reference.yui-compressor-ant-task.jar}" /> <path id="task.classpath"> <pathelement location="${yui-compressor.jar}" /> <pathelement location="${yui-compressor-ant-task.jar}" /> </path> <!-- yui-compressor task definition --> <taskdef name="yui-compressor" classname="net.noha.tools.ant.yuicompressor.tasks.YuiCompressorTask"> <classpath refid="task.classpath" /> </taskdef> <!-- concatenation of javascript --> <echo message="Building global javascript" /> <concat destfile="${build.dir}/web/common/js/global.js" force="no"> <!-- explicitly order js concat because ordering matters here --> <fileset dir="${build.dir}" includes="web/common/js/jquery.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.bgiframe.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.hoverIntent.js" /> <fileset dir="${build.dir}" includes="web/common/js/ui.core.js" /> <fileset dir="${build.dir}" includes="web/common/js/ui.autocomplete.js" /> <fileset dir="${build.dir}" includes="web/common/js/ui.datepicker.js" /> <fileset dir="${build.dir}" includes="web/common/js/ui.tabs.js" /> <fileset dir="${build.dir}" includes="web/common/js/tablesort.js" /> <fileset dir="${build.dir}" includes="web/common/js/customsort.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.blockUI.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.form.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.ifixpng.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.superfish.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.cluetip.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.scrollTo.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.jqModal.js" /> <fileset dir="${build.dir}" includes="web/common/js/validation.js" /> <fileset dir="${build.dir}" includes="web/common/js/jquery.timeentry.js" /> </concat> <!-- concatenation of cascading stylesheets --> <echo message="Building global cascading stylesheets" /> <concat destfile="${build.dir}/web/common/css/global.css" force="no"> <fileset dir="${build.dir}" includes="web/common/css/base.css" /> <fileset dir="${build.dir}" includes="web/common/css/superfish.css" /> <fileset dir="${build.dir}" includes="web/common/css/announcements.css" /> <fileset dir="${build.dir}" includes="web/common/css/myvt.css" /> <fileset dir="${build.dir}" includes="web/common/css/forms.css" /> <fileset dir="${build.dir}" includes="web/common/css/cluetip.css" /> <fileset dir="${build.dir}" includes="web/common/css/tables.css" /> <fileset dir="${build.dir}" includes="web/common/css/ui.tabs.css" /> <fileset dir="${build.dir}" includes="web/common/css/ui.datepicker.css" /> <fileset dir="${build.dir}" includes="web/common/css/ui.autocomplete.css" /> <fileset dir="${build.dir}" includes="web/common/css/linkspan.css" /> <fileset dir="${build.dir}" includes="web/common/css/stepMenu.css" /> <fileset dir="${build.dir}" includes="web/common/css/print.css" /> <fileset dir="${build.dir}" includes="web/common/css/youHaveMessages.css" /> </concat> <!-- invoke compressor --> <yui-compressor warn="false" charset="UTF-8" fromdir="${build.dir}" todir="${build.dir}"> <include name="web/common/js/global.js" /> <include name="web/common/css/global.css" /> </yui-compressor> </target> <!-- * purge-src takes our compressed files, moves them to the base /common dir * and deletes the source js and css dirs from the build dir --> <target name="purge-src" depends="minify"> <echo message="Purging javascript and stylesheet sources" /> <move file="${build.dir}/web/common/js/global-min.js" tofile="${build.dir}/web/common/global.js"/> <move file="${build.dir}/web/common/css/global-min.css" tofile="${build.dir}/web/common/global.css"/> <delete dir="${build.dir}/web/common/js" /> <delete dir="${build.dir}/web/common/css" /> </target>
What you see there is essentially four steps.
- Concatenate all the constituent source files into a
global.jsand aglobal.css - Compress both of these files, which creates
global-min.jsandglobal-min.css(default behavior) - Move these files out of the source directories and into the root of the common web directory as
global.jsandglobal.css - Delete the source directories in our build folder so they don’t get deployed with the web archive
Because certain browsers (IE) break without explicit ordering, we unfortunately can’t just use “*.js” and “*.css” in our concatenation step, but having to explicitly list our components in the build file certainly isn’t the end of the world. The nice thing is that the Ant task will even print out handy statistics on just how much you’ve been able to compress the files down. In our case, we have about 441.8KB of common Javascript and CSS in our source code that, by the time it gets sent to the user, has been minified and/or gzipped to about 89KB.