Previously, I’ve casually benchmarked the Javascript speeds of 64-bit browsers v. their 32-bit counterparts (here); more recently, I benchmarked a 64-bit compile of FLAC against several other 32-bit compiles of the same version (here).

This time, I decided to test various and sundry file compression utilities—more specifically, those which offer both 32- and 64-bit versions of themselves. This benchmark did not exhaustively test all potential combinations of compression options (if you’re interested in that, see Werner Bergman’s excellent Maximum Compression and Matt Mahoney’s Data Compression Programs), nor will it compare various compressors to each other; neither will it even list how well the programs actually compressed, since that’s not really a consideration here. The sole purpose of the benchmark was to compare the execution time of a 32-bit program with its 64-bit version.

The corpus in this case was enwiki9, compressed to a RAM disk to minimize the potential effects of write latency. I wanted the corpus to be sufficiently large to better tease out significant differences in these compressors over a large dataset.

The results for each compressor are listed on their own page, as well as an explanation of the compressor, its origin, and any additional notes. One notable exception to this list is WinRK, which is available in a 64-bit version but contains no command-line interface.

I decided to do some quick and dirty benchmarks to track the progress of LZMA/7-Zip over time. I went back as far as Igor supplied binaries, including one from the very old 3.x series. Rather than test every single release between then and now, I used only “stable” releases, with the exception of version 4.65, which is the latest version of any sort, as well as 4.66, which uses an alpha version of Igor’s new LZMA2 codec (and, as you’ll see, provides definite performance improvement).

I used Igor’s Timer utility to time the process (global time was reported). The corpus in this case was the Linux kernel source, v2.6.28. I conducted these tests on a RAM disk to eliminate hard disk latency issues (especially for decompressions, which improved by about 25% from my initial HDD-based tests). My rig is a Intel Core 2 Quad Q6600 [2.4Ghz], with 4GB of RAM (one dedicated to the RAM disk), running Vista SP1 x64.

The command line setup was an approximation of the 7-Zip GUI’s “ultra” settings: -t7z -m0=lzma -mx=9 -mfb=64 -md=32m -ms=on, letting the archiver auto-choose the number of threads to spawn.

The Data

The Analysis

Without conducting a more thorough battery of tests on a variety of different configurations, it’s difficulty to say with certain just where the performance improvements came from, be it better using of threading or multiprocessors, general algorithmic improvements, or something else. I also don’t know if the performance increases we see reside in improvements to LZMA itself as Igor was finalizing it, or just the code quality of 7-Zip, which implements LZMA.

In any case, the improvements since 3.13 are very clear (remember that lower is better), at least for compression, and for “ultra” settings. Decompression remained largely similar, which surprised me. Some of these results might be directly tied to the number and type of files that were compressed in the case: 4.66, for instance, improves decompression speed for uncompressable files, but no such files exist here since it’s source code.

Hats off to Igor Pavlov for his steady improvement on both a really great compression standard and one of my favorite pieces of software for Windows.

After the initial learning curve, this has all been going swimmingly. I merged my first development branch into the trunk yesterday, and this branch just so happens to dovetail nicely into the whole point of this post, which is the YUI compressor, an open-source javascript and CSS minification tool developed by Yahoo’s YUI team.

The Problem

To understand quickly why one should minify production client-side code, consider only that with the upward trend of size in javascript libraries (and the necessary files for such libraries), it’s possibly to be downloading a lot of client-side code in a typical web application, especially as its scope grows.

For a long time, I was using Dean Edward’s Packer, as was everyone, because its Base62 encoding produced the very lowest file size. However, what should have been obvious to everyone is that eval(bunch_of_stuff_goes_here) is making the browser do a lot more work, and this is a problem on dinosaurs like IE6.

To make matters worse, the nature of such encoding also meant that for servers which tried to compress outgoing content like javascript (either with zlib or gzip), the compression ratio suffered. Just look at this table that Julien Lecomte posted last August.

I said to myself, “Hey, we use a lot of this stuff, and we still support a lot of users with slow computers and slow browsers.” So, I moved our project from a Packer-based compression to a YUI-based compression, and turned on server-side GZIP compression for javascript files. The only problem was that I was storing the javascript files already minified, and simply pasted them into a large a couple of large global .js files. I had to keep a separate source directory, along with any customizations.

This got to be a pain in the ass, as you might well expect, and so when it occurred to me that I might be able to use the YUI library at build-time in our Netbeans project, I immediately sprung into action.

This was around April, and one night while attending Sungard Summit in Anaheim. I did the initial work to get our Netbeans project into a state that could use the YUI compressor at build-time, creating separate source file directories and breaking our massive javascript file into modules; I did the same with our CSS, splitting it up based on what it decorated.

There are a few tutorials about using the YUI library. Some of them involve adding the YUI library to Ant’s classpath (didn’t want to go down this route); a lot of them involve invoking the library as an external executable during the build process, which is messy.

The solution I finally settled on was yui-compressor-ant-task, a small library that allows Ant to use YUI as a build task. By adding this library and the YUI compressor library to our common libraries folder, and enabling them at build only (and not for deploying in the web archive), it makes using the compressor pretty easy.

Here’s part of our build.xml:

<!--
   * minify will concatenate all of our non-TinyMCE javascripts and stylesheets
   * then use the YUI compressor library to compress them
   -->
   <target name="minify">
       <!--${libs} is path to the downloaded jars -->
       <property
           name="yui-compressor.jar"
           location="${file.reference.yuicompressor.jar}" />
       <property
           name="yui-compressor-ant-task.jar"
           location="${file.reference.yui-compressor-ant-task.jar}" />
 
       <path id="task.classpath">
           <pathelement location="${yui-compressor.jar}" />
           <pathelement location="${yui-compressor-ant-task.jar}" />
       </path>
 
       <!-- yui-compressor task definition -->
       <taskdef
           name="yui-compressor"
           classname="net.noha.tools.ant.yuicompressor.tasks.YuiCompressorTask">
 
           <classpath refid="task.classpath" />
       </taskdef>
 
       <!-- concatenation of javascript -->
       <echo message="Building global javascript" />
       <concat destfile="${build.dir}/web/common/js/global.js" force="no">
           <!-- explicitly order js concat because ordering matters here -->
           <fileset dir="${build.dir}" includes="web/common/js/jquery.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.bgiframe.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.hoverIntent.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.core.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.autocomplete.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.datepicker.js" />
           <fileset dir="${build.dir}" includes="web/common/js/ui.tabs.js" />
           <fileset dir="${build.dir}" includes="web/common/js/tablesort.js" />
           <fileset dir="${build.dir}" includes="web/common/js/customsort.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.blockUI.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.form.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.ifixpng.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.superfish.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.cluetip.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.scrollTo.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.jqModal.js" />
           <fileset dir="${build.dir}" includes="web/common/js/validation.js" />
           <fileset dir="${build.dir}" includes="web/common/js/jquery.timeentry.js" />
       </concat>
 
       <!-- concatenation of cascading stylesheets -->
       <echo message="Building global cascading stylesheets" />
       <concat destfile="${build.dir}/web/common/css/global.css" force="no">
           <fileset dir="${build.dir}" includes="web/common/css/base.css" />
           <fileset dir="${build.dir}" includes="web/common/css/superfish.css" />
           <fileset dir="${build.dir}" includes="web/common/css/announcements.css" />
           <fileset dir="${build.dir}" includes="web/common/css/myvt.css" />
           <fileset dir="${build.dir}" includes="web/common/css/forms.css" />
           <fileset dir="${build.dir}" includes="web/common/css/cluetip.css" />   
           <fileset dir="${build.dir}" includes="web/common/css/tables.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.tabs.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.datepicker.css" />
           <fileset dir="${build.dir}" includes="web/common/css/ui.autocomplete.css" />
           <fileset dir="${build.dir}" includes="web/common/css/linkspan.css" />
           <fileset dir="${build.dir}" includes="web/common/css/stepMenu.css" />
           <fileset dir="${build.dir}" includes="web/common/css/print.css" />
           <fileset dir="${build.dir}" includes="web/common/css/youHaveMessages.css" />
       </concat>
 
       <!-- invoke compressor -->
       <yui-compressor warn="false" charset="UTF-8" fromdir="${build.dir}" todir="${build.dir}">
           <include name="web/common/js/global.js" />
           <include name="web/common/css/global.css" />
       </yui-compressor>
 
   </target>
 
   <!--
   * purge-src takes our compressed files, moves them to the base /common dir
   * and deletes the source js and css dirs from the build dir
   -->
   <target name="purge-src" depends="minify">
       <echo message="Purging javascript and stylesheet sources" />
 
       <move file="${build.dir}/web/common/js/global-min.js" tofile="${build.dir}/web/common/global.js"/>
       <move file="${build.dir}/web/common/css/global-min.css" tofile="${build.dir}/web/common/global.css"/>
 
       <delete dir="${build.dir}/web/common/js" />
       <delete dir="${build.dir}/web/common/css" />
 
   </target>

What you see there is essentially four steps.

1. Concatenate all the constituent source files into a global.js and a global.css
2. Compress both of these files, which creates global-min.js and global-min.css (default behavior)
3. Move these files out of the source directories and into the root of the common web directory as global.js and global.css
4. Delete the source directories in our build folder so they don’t get deployed with the web archive

Because certain browsers (IE) break without explicit ordering, we unfortunately can’t just use “*.js” and “*.css” in our concatenation step, but having to explicitly list our components in the build file certainly isn’t the end of the world. The nice thing is that the Ant task will even print out handy statistics on just how much you’ve been able to compress the files down. In our case, we have about 441.8KB of common Javascript and CSS in our source code that, by the time it gets sent to the user, has been minified and/or gzipped to about 89KB.

I ran a brief test using generated Lorem Ipsum text in approximate amounts (the leftmost column), and recorded its size (in bytes) when pasted into Notepad, and then as OpenDocument Text (OpenOffice.org 2.3.1), and then as OOXML (Office 2007 SP1).

After the data table is a graphical representation of the results. It’s clear that ODF slips below OOXML somewhere between 300Kb and 400Kb of raw textual data.

Here’s the catch, though: the quality of a compressor isn’t measured by its final compression ratio. The PAQ series of a compressors, for instance, offer great compression and really, truly awful compression times. The time goes with the highest compression levels of WinRK (a proprietary Win32 format with an accompanying GUI). But disk is cheap: nobody really cares about a fraction of a percentage of compression efficiency, do they? What people really want is for their (inevitable) archiving GUI to take less time doing what it does.

In this spirit, I have compiled not so much an exhaustive less of possible compression algorithms (I’ll leave that to Werner, who is very good at what he does), but rather a short list of the most common formats, tested on three different (relatively well-known) corpuses: the Calgary Corpus, the newer Canterbury Corpus, and Andrew Tridgell’s 1999 Large Corpus. The first of these two are corpuses used to test the very kind of academic project which I’ve avoided. I dislike using them because they are small in size, which means that there is significantly less opportunity for variations in compression formats to manifest themselves. In the interest of verifiability, however, I have used them. I also included Andrew Tridgell’s large corpus because it’s been my experience that small test corpuses tend to vary too much too to disk I/O latency and other vagaries of compression algorithms.

What will follow is a data table for each corpus, followed by some brief observations about each.

First, a note about the test environment:

• Windows Vista x64
• Intel Q6600 Quad-Core
• 4GB Corsair PC2 6400
• Western Digital Caviar WD1600YS SATAII, 160GB (system drive)
• Timer: Igor Pavlov’s timer.exe (times reported are “Process” times).

Next, a note about compressor versions

Now on to the benchmarks…

The Calgary Corpus dates back to the late 80s. It’s become the test to perform, but it may or may not adequately represent the standard compressor workload in 2008. You’ll notice that Winrar’s maximum setting produces the smallest archive, and more quickly than the neighboring 7-zip runs. Notice, too, that among the lowest values, there tends to be a sort of “bottoming-out” point at which the speed of the compressor’s process in CPU is limited by the speed of the disk.

I’m still not entirely able to figure out the Canterbury Corpus; it’s ostensibly an “update” to the aging Calgary Corpus. One would think that having been created more than a decade after it’s predecessor, and with the express purpose of more accurately representing the compressor workload of 2001, it would at least be larger (hard disks and file sizes have increased in size since 1989, believe it or not), but in fact it’s not, which was somewhat of a disappointment to me, as I saw entirely the same trends as with the previous corpus. Is that an accurate determination of the algorithms in question? Maybe not—read on.

Mostly interestingly in the Tridgell’s “large-corpus,” we finally see 7-Zip spring ahead of WinRAR in terms of pure compression ratio (and in speed, too, in some cases). I’m not an expert on compression, so I can’t tell you why certain efficiencies only manifest themselves over large datasets, but clearly 7-Zip wins in more modern cases where large data-sets (mostly text, if Tridgell’s description is accurate) are present.

Clearly, the LZMA algorithm (the heart of 7-Zip) is something to be proud of; not only is it GPL, but it often outperform the popular WinRAR in both pure compression and in efficiency as well. I’m a little surprised that the 7-Zip *nix port, p7zip, hasn’t gained more traction in Linux, but I suppose that old ways die hard. The cheapness of disk and bandwidth nowadays rather point to more transparent compression as the ideal rather than whatever archiving format has the best compression in terms of purely numeric results.

For those of you looking for a decent free arching program, check 7-Zip out; for those of you who lust after data tables of compression benchmarks, give Werner’s a look: it’ll satiate your desire for tabular results in ways you never thought possible.

In my lust for benchmarking compressors and things of that sort, I decided to attack some typical Linux CLI compressors for a very general comparison of their relative efficiency.

If you want to skip straight to the results, go ahead and view the Linux CLI compression benchmarks in PDF format.

First, however, I need to explain my corpus, compressors, and methodology.

Test Information

Corpus

OSDB is a database from a benchmarking project on SourceForge used to test open-source database performance. The version I downloaded was v0.21

linux-2.6.19 is, I hardly need say, the source code for version 2.6.19 of the Linux kernel

final_fantasy_tactics is a binary dump of the Playstation disc for Final Fantasy Tactics, a good mix of video, audio, and executable code.

Webster’s 1913 dictionary is also just what it sounds like. I pulled it from Project Gutenberg as one file, but it appears in the meantime to have disappeared, or at least been split up by letter.

Compressors (official packages from Ubuntu 6.10 repositories)

tar (1.15.91) is not a compressor, but is rather an archiver, used to gather multiple files together into a file which can then be compressed. I made tarballs out of everything for consistency’s sake

gzip (1.3.5) is a very lightweight compressor used in large part for transparent compression across networks (the page you’re reading at this very moment was probably gzipped before it was sent to your browser, which then unpacked it)

bzip2 (1.0.3) is a powerful block-sorting compressor—the bigger, stronger, slower brother of gzip.

zip (2.32) is probably the most common compression format in Windows. There are a number of different implementations, but this is the standard version that comes in any GNU toolchain. It uses the common DEFLATE method.

rzip (2.1) is a modified version of bzip2, the advantage of which lies in its 900MB history buffer, meaning that it can take advantage of data redundancies over much longer distances than its brethen. Its principle advantage is apparent for large files.

7za (4.42) is a Unix port of the esteemed 7-Zip archiver for Windows. It has several different modes for compression for its 7z format (PPMd and LZMA, the latter being principle), of which two were tested (the third is bzip2, which is already being tested).

rar (3.60) is the non-free command-line version of the famous WinRAR archiver.

arj (3.10.22) was a very popular archiving in the early-to-mid 90s, alongside pkzip. It isn’t of much use now, but exists as Free Software in any decent Linux repo, and so was also tested.

lha (1.14i) is an open-source compressor of the LZH format, an early format (1988) that has also fallen out of vogue. It has no switches or useful options.

lzop (1.01) is a free implementation of the super-fast LZO algorithm, providing very basic compression very quickly. It has been surpassed in efficiency by QuickLZ, but the latter didn’t have a Unix port by the time of this test

zoo (2.10) is a compression format from the 1980s that was mostly popular with OpenVAX servers. It is not seriously used anymore except perhaps on legacy machines.

Methodology

Any relevant switches have been documented in the data table. All compressions were run from the bash shell. I created a script for each compressor that compressed and then decompressed the file for each setting in turn. It was invoked with the GNU time which then wrote the utilized time to a log file. After each compression, an ls -lrt command was invoked, to get the precise size of the compressed archive, and this was also written to the log file. This was done on a Ubuntu 6.10 system, in the GNOME environment. Any other programs in use were in use for the duration of the test, so consistency should not be an issue.

Data

The table itself is too big to display within the confines of my blog, so you’ll need to view it separately. Linux CLI compression benchmarks, PDF format

Analysis

What conclusions can be drawn from these benchmarks? Linux’s relative paucity of readily-available compressors isn’t really a hindrance to it—there are few actually useful formats here: gzip and bzip2, obviously, are the foundation upon which Linux compression is built. zip offers no advantages over gzip except that it is both an archiver and a compression (a point made largely moot by tar and pipes, e.g. tar -czf foo.tar.gz bar/), but it is good to have for compatibility purposes. rzip, while interesting, is useful only for large files: if you’re attempting to squeeze every last ounce of extra space from a file, it would probably behoove you to use a different compressor, anyway.

In terms of the heavy-duty archivers like 7-Zip and RAR, they both offer quite a bit of power at the expense of being either difficult to invoke (7z creation is still broken in file-roller on Ubuntu systems) or non-free (rar is technically shareware, and not readily available in all distributions). It is interesting to note that 7-Zip still spanks WinRAR in general performance. With regard to 7-Zip, I’ve concluded that PPMd is not generally a worthwhile algorithm to use: in some cases, it offers a modest increase in efficiency over LZMA, but it’s decompression times are horrible, generally equal to or greater than its compression times. LZMA, meanwhile, offers modest compression times (relative to its ratio) and lightning-fast decompression. One odd behavior is that compression efficiency seems to work on plateaus—with LZMA, for instance, there are only 3 different compression ratios for 9 switches (1-4, 5-6, and 7-9).

lzop, or even LZO compression is general, doesn’t offer much in the way of compression improvement over a gzip on default settings, though it does offer a slight improvement in decompression.

Other, more esoteric formats like arj, zoo, and lha are really no more than historical curiosities. The only compressor from these than approaches the capabilities of gzip or bzip2 is lha, at which point why even bother with the latter?

Caveats

If you look at the data closely, you’ll notice some strange behavior, especially regarding the first test in a series. At first, I believed that the least-intensive compressions tended to be limited by disk I/O, rather than CPU cycles. Eventually, however, I began to wonder if it had more to do with reading the file into memory than it did with file writes. Running in a script, the original file might stay cached after it was read in the first time, speeding up additional compressions by not having to read the file from the disk again. Take from it what you will: these are very much unofficial benchmarks.