Using YUI compressor in a web project

Last year, I moved our small programming department from using JDeveloper and editing shared files directly on a network drive¹ to using Netbeans 6.x and a proper version control system (Subversion).

After the initial learning curve, this has all been going swimmingly. I merged my first development branch into the trunk yesterday, and this branch just so happens to dovetail nicely into the whole point of this post, which is the YUI compressor, an open-source javascript and CSS minification tool developed by Yahoo’s YUI team.

The Problem

To understand quickly why one should minify production client-side code, consider only that with the upward trend of size in javascript libraries (and the necessary files for such libraries), it’s possibly to be downloading a lot of client-side code in a typical web application, especially as its scope grows.

For a long time, I was using Dean Edward’s Packer, as was everyone, because its Base62 encoding produced the very lowest file size. However, what should have been obvious to everyone is that eval(bunch_of_stuff_goes_here) is making the browser do a lot more work, and this is a problem on dinosaurs like IE6².

To make matters worse, the nature of such encoding also meant that for servers which tried to compress outgoing content like javascript (either with zlib or gzip), the compression ratio suffered. Just look at this table that Julien Lecomte posted last August.

Javascript compression
File	File size (bytes)	Gzipped file size (bytes)
Original jQuery libraries	62,885	19,758
jQuery minified with JSMin	36,391	11,541
jQuery minified with Packer	21,557	11,119
jQuery minified with the YUI compressor	31,822	10,818

I said to myself, “Hey, we use a lot of this stuff, and we still support a lot of users with slow computers and slow browsers.” So, I moved our project from a Packer-based compression to a YUI-based compression, and turned on server-side GZIP compression for javascript files. The only problem was that I was storing the javascript files already minified, and simply pasted them into a large a couple of large global .js files. I had to keep a separate source directory, along with any customizations.

This got to be a pain in the ass, as you might well expect, and so when it occurred to me that I might be able to use the YUI library at build-time in our Netbeans project, I immediately sprung into action.

This was around April, and one night while attending Sungard Summit in Anaheim. I did the initial work to get our Netbeans project into a state that could use the YUI compressor at build-time, creating separate source file directories and breaking our massive javascript file into modules; I did the same with our CSS, splitting it up based on what it decorated.

There are a few tutorials about using the YUI library. Some of them involve adding the YUI library to Ant’s classpath (didn’t want to go down this route); a lot of them involve invoking the library as an external executable during the build process, which is messy.

The solution I finally settled on was yui-compressor-ant-task, a small library that allows Ant to use YUI as a build task. By adding this library and the YUI compressor library to our common libraries folder, and enabling them at build only (and not for deploying in the web archive), it makes using the compressor pretty easy.

Here’s part of our build.xml:

<!--
   * minify will concatenate all of our non-TinyMCE javascripts and stylesheets
   * then use the YUI compressor library to compress them
   -->
   <target name="minify">
       <!--${libs} is path to the downloaded jars -->
       <property
           name="yui-compressor.jar"
           location="${file.reference.yuicompressor.jar}" />
       <property
           name="yui-compressor-ant-task.jar"
           location="${file.reference.yui-compressor-ant-task.jar}" />
 
       <path id="task.classpath">
           <pathelement location="${yui-compressor.jar}" />
           <pathelement location="${yui-compressor-ant-task.jar}" />
       </path>
 
       <!-- yui-compressor task definition -->
       <taskdef
           name="yui-compressor"
           classname="net.noha.tools.ant.yuicompressor.tasks.YuiCompressorTask">
 
           <classpath refid="task.classpath" />
       </taskdef>
 
       <!-- concatenation of javascript -->
       <echo message="Building global javascript" />
       <concat destfile="${build.dir}/web/common/js/global.js" force="no">
           <!-- explicitly order js concat because ordering matters here -->
           <fileset dir="${build.dir}" includes="web/common/js/jquery.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.bgiframe.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.hoverIntent.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.core.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.autocomplete.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.datepicker.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.tabs.js" />
           <fileset dir="${build.dir}" includes="web/common/js/tablesort.js" />
           <fileset dir="${build.dir}" includes="web/common/js/customsort.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.blockUI.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.form.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.ifixpng.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.superfish.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.cluetip.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.scrollTo.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.jqModal.js" />
           <fileset dir="${build.dir}" includes="web/common/js/validation.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.timeentry.js" />
       </concat>
 
       <!-- concatenation of cascading stylesheets -->
       <echo message="Building global cascading stylesheets" />
       <concat destfile="${build.dir}/web/common/css/global.css" force="no">
           <fileset dir="${build.dir}" includes="web/common/css/base.css" />
           <fileset dir="${build.dir}" includes="web/common/css/superfish.css" />
           <fileset dir="${build.dir}" includes="web/common/css/announcements.css" />
           <fileset dir="${build.dir}" includes="web/common/css/myvt.css" />
           <fileset dir="${build.dir}" includes="web/common/css/forms.css" />
           <fileset dir="${build.dir}" includes="web/common/css/cluetip.css" />   
           <fileset dir="${build.dir}" includes="web/common/css/tables.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.tabs.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.datepicker.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.autocomplete.css" />
           <fileset dir="${build.dir}" includes="web/common/css/linkspan.css" />
           <fileset dir="${build.dir}" includes="web/common/css/stepMenu.css" />
           <fileset dir="${build.dir}" includes="web/common/css/print.css" />
           <fileset dir="${build.dir}" includes="web/common/css/youHaveMessages.css" />
       </concat>
 
       <!-- invoke compressor -->
       <yui-compressor warn="false" charset="UTF-8" fromdir="${build.dir}" todir="${build.dir}">
           <include name="web/common/js/global.js" />
           <include name="web/common/css/global.css" />
       </yui-compressor>
 
   </target>
 
   <!--
   * purge-src takes our compressed files, moves them to the base /common dir
   * and deletes the source js and css dirs from the build dir
   -->
   <target name="purge-src" depends="minify">
       <echo message="Purging javascript and stylesheet sources" />
 
       <move file="${build.dir}/web/common/js/global-min.js" tofile="${build.dir}/web/common/global.js"/>
       <move file="${build.dir}/web/common/css/global-min.css" tofile="${build.dir}/web/common/global.css"/>
 
       <delete dir="${build.dir}/web/common/js" />
       <delete dir="${build.dir}/web/common/css" />
 
   </target>

What you see there is essentially four steps.

Concatenate all the constituent source files into a global.js and a global.css
Compress both of these files, which creates global-min.js and global-min.css (default behavior)
Move these files out of the source directories and into the root of the common web directory as global.js and global.css
Delete the source directories in our build folder so they don’t get deployed with the web archive

Because certain browsers (IE) break without explicit ordering, we unfortunately can’t just use “*.js” and “*.css” in our concatenation step, but having to explicitly list our components in the build file certainly isn’t the end of the world. The nice thing is that the Ant task will even print out handy statistics on just how much you’ve been able to compress the files down. In our case, we have about 441.8KB of common Javascript and CSS in our source code that, by the time it gets sent to the user, has been minified and/or gzipped to about 89KB.

Executive summary: “Gah!”[↩]
In fairness to Dean, his Packer has a non-obfuscating mode which works just like other minifiers. Everyone latched onto the Base62 mode because it created the smallest files without worrying about Apache compression.[↩]

3 Comments by Ben / September 22, 2008 / Posted in: general

Tagged: compression, css, Internet Explorer, Java, javascript, jquery, web design, work

3 ResponsesLeave one →

Dean
/ Wednesday, May 13th, 2009

I can not get this code to work for me. I have the following set up:
mySite
+– Build
+– css
+– js
+– pages
+– Website
+– css
+– js
+– pages
Cots

I am trying to take my js files from under ‘Website’ and minify and compress them into the ‘Build/js’ drirectory… however I keep getting ‘No files found’ echo prints.

Reply
- Ben
  / Wednesday, May 13th, 2009
  
  You’ll have to be a little clearer with your site layout. And can you post your build file on PasteBin or something?
  
  Reply
Armindo Cachada
/ Monday, June 15th, 2009

In my blog you can find an example on how to use YUI compressor in an ant task. This code also has a nice feature that concatenates css and js files without hardcoding them into the build.xml file, using a custom ant task.

You can find more information at:

http://simplyolaf.blogspot.com/2009/06/minifying-css-and-js-with-yahoo.html

and

http://simplyolaf.blogspot.com/2009/06/minimizing-number-of-http-requests-by.html

Reply

Recent Comments
- Andy
  Nice review, Ben. I actually hadn’t heard of this book, but I might have to look for it. True SF...
- Ben
  If you mean the current, theme, I can’t take the credit for that. It’s Renegade, by...
- Clare Swindlehurst
  I’ve just stumbled upon your site and wanted to tell you how great I think your...
- mike
  just found out about the other book myself, waiting for it to be delivered. i dont blame you for...
- Charles
  It’s interesting that the filmmakers (or the screenwriter) made the decision to approach the...
Now Reading
- Planned books:
  - House of Leaves by Mark Z. Danielewski
  - Solaris by Stanislaw Lem
  - The Family: The Secret Fundamentalism at the Heart of American Power by Jeff Sharlet
  - Stuff: Compulsive Hoarding and the Meaning of Things by Prof. Randy Frost Ph.D., Prof. Gail Steketee Ph.D.
  - Hitch-22: A Memoir by Christopher Hitchens
  - Confessions of an Ugly Stepsister by Gregory Maguire
  - To Your Scattered Bodies go by Philip Jose Farmer
  - And God Said: How Translations Conceal the Bible’s Original Meaning by Joel M. Hoffman
  - I Love It When You Talk Retro: Hoochie Coochie, Double Whammy, Drop a Dime, and the Forgotten Origin by Ralph Keyes
  - The Possessed: Adventures with Russian Books and the People Who Read Them by Elif Batuman
- Current books:
  - Oil Panic and the Global Crisis: Predictions and Myths by Steven M. Gorelick
  - Generosity: An Enhancement by Richard Powers
  - The Great Influenza: The story of the deadliest pandemic in history by John M. Barry
- Recent books:
  - The Forever War by Joe Haldeman
  - Juliet, Naked: a novel by Nick Hornby
  - Shades of Grey: A Novel by Jasper Fforde
  - The Unnamed by Joshua Ferris
  - K-Pax by Dr. Gene Brewer
Tags

52 Books in 52 Weeks Bill Bryson books Christopher Hitchens css economics essays etymology fantasy fiction Firefox FLAC Friday Random Ten history humor javascript legal linguistics Linux literature memes Middle East miscellany movies music nonfiction open source personal philosophy poetry politics psychology religion reviews satire science science fiction site updates social sociology software technology web design Wednesday's Word Windows

A Modest Construct