On the heels of my recent comparison of general-purpose data compressors, I bring you the results of far too much time spent hunched in front of a command line, calculating ratios.

The benchmark’s ostensible purpose, at least at first, was to compare four mainstream codecs—FLAC, Monkey’s Audio, WavPack, and OptimFROG—and see how their current stable versions stacked up with the version in development. I ended up throwing in a few others—TTA, LA, and Shorten—for comparative purposes. This test looked at Encoding Time and Compression Ratio; decoding speed was not tested.

If you don’t want to look at the raw data, I’ll save you the trouble of even clicking past the fold.

FLAC
FLAC has languished at v1.1.2 since February of 2005. It has the uncomfortable position of having reached an stable position in the world of lossless codecs, even garnering some hardware support. As a result, it can’t make any sweeping changes without breaking compatibility or hurting its (excellent) decoding speed.
The result is that there have been some definite improvements in compression, though nothing that will blow anyone away. There appears to be an album-wide net loss of between 0.5 and 1.5%, enough to shave another few megabytes from the total size. The default compression (-5) for 1.1.3b2 now compresses better than the maximum practical compression (-8) of 1.1.2. Of course, part of 1.1.3’s appeal is other features, like album art embedding, but that’s beyond the scope of my test.
WavPack
WavPack has been around for a long time (it supposedly inspired Matt Ashland, the creator of Monkey’s Audio, to work on his own), but for some reason has always been underappreciated in the world of lossless compressors. I suppose that’s because it manages to fall somewhere in the middle of Monkey’s Audio and FLAC, not offering compression ratios as low as the former or decode speeds as high as the latter. Still, I was impressed that WavPack not only encoded faster than FLAC, but it got a better compression ratio, too—and it’s decoding speed is also very excellent.
WavPack has two settings: default and -h, or high. It also has a -x[1-6] switch that can possibly shave off a fraction of the size at the expense of a really long encode time. The tendency of 4.4a3 under the default setting was to take a few seconds longer to encode with the benefit of between 0.1% and 1.5% improvement in the compression. For the high setting, however, the encoding time was a few seconds quicker, but the ratio was always worse.
Monkeys Audio (APE)
Monkey’s Audio has a storied history, part of which has to do with its laughable “open source” license, spurned by every Linux distribution I’ve ever seen. APE offers better compression than FLAC without drastically increased encoding times, but it’s particularly CPU-intensive for decoding, making it impractical for handhelds or DAPs.
The change from 3.99 to 4.01b2 has been entirely in the GUI frontend. The encoding times weren’t different enough to be significant, and the sizes were exactly the same. My understanding, though, is that it’s been optimized for dual-core processors, so if you’ve got one, you’ll see a significantly faster encode time.
OptimFROG
OptimFROG is the only truly closed-source encoder in the primary four. It offers the best compression of them as well, but also long encode times and piss-poor playback speeds. It hasn’t gained very much traction except as a curiosity: its closed source nature prevents its use in Linux distros, and its intense resource usage means it will likely never be supported in hardware.
The changes in OptimFROG between 4.520b1 (stable) and 4.600ex (testing) aren’t in the default encoding options. The —highnew mode (the —bestnew mode took 100 minutes for the first album, so I very quickly decided to try the one below it), which increased encoding time for modest compression gains under 0.5%.
Others
TrueAudio is another open-source format that’s languished in obscurity. It’s resolutely middle-of-the-road, offering decent compression and a fast speed. Unfortunately, that means it also gets shown up by the Best in Class codecs.
Shorten was the very first of the serious lossless encoders, first showing up in 1993. It hasn’t gone anywhere in years, and even the organization most attached it to (the live music trading site, ETree, long ago replaced it with FLAC. It offers super-fast decoding speed, but the very worst compression.
LA is another closed-source encoder with very slow encoding times (not quite as bad as OptimFROG) that repeatedly trounces every other codec available in terms of compression ratio. However, its high resource usage and closed nature make it impractical for just about everything but a curio. It also hasn’t seen any development in some time now.

To see the complete table, look below the fold.

See this table in a more attractive PDF format

Nine Inch Nails :: The Fragile (Left)
Codec Version Mode Encoding Time (s) Size (b) Ratio
wav 0.000 581,673,164 1.000
flac 1.1.2 -5 55.875 355,413,878 0.611
flac 1.1.2 -8 273.750 353,897,562 0.608
flac 1.1.3b2 -5 55.875 352,415,056 0.606
flac 1.1.3b2 -8 272.515 350,226,095 0.602
wavpack 4.31 40.281 352,381,226 0.606
wavpack 4.31 -h 62.219 346,201,250 0.595
wavpack 4.4a3 43.312 350,619,410 0.603
wavpack 4.4a3 -h 54.656 347,538,278 0.597
ape 3.99 -c2000 65.657 341,664,748 0.587
ape 3.99 -c4000 139.453 338,903,128 0.583
ape 4.01b2 -c2000 65.281 341,664,748 0.587
ape 4.01b2 -c4000 138.281 338,903,128 0.583
optimfrog 4.560b1 –mode normal 174.156 339,088,247 0.583
optimfrog 4.560b1 –mode highnew 842.797 335,939,936 0.578
optimfrog 4.600ex –mode normal 183.516 339,088,247 0.583
optimfrog 4.600ex –mode highnew 881.016 335,376,237 0.577
tta 3.3 44.969 348,038,698 0.598
shorten 3.60 62.094 383,321,713 0.659
la 0.4b 546.609 334,871,479 0.576
la 0.4b -high 735.265 333,839,758 0.574
Esmerine :: If Only a Sweet Surrender to the Nights to Come Be True
Codec Version Mode Encoding Time (s) Size (b) Ratio
wav 0.000 534,637,868 1.000
flac 1.1.2 -5 53.565 271,813,492 0.508
flac 1.1.2 -8 245.594 270,725,001 0.506
flac 1.1.3b2 -5 49.906 268,373,295 0.502
flac 1.1.3b2 -8 246.594 266,655,680 0.499
wavpack 4.31 36.000 268,663,814 0.503
wavpack 4.31 -h 54.328 263,620,948 0.493
wavpack 4.4a3 37.796 267,954,980 0.501
wavpack 4.4a3 -h 47.172 265,285,296 0.496
ape 3.99 -c2000 58.875 257,841,484 0.482
ape 3.99 -c4000 126.266 251,692,756 0.471
ape 4.01b2 -c2000 59.078 257,841,484 0.482
ape 4.01b2 -c4000 125.297 251,692,756 0.471
optimfrog 4.520b1 –normal 160.734 254,785,551 0.477
optimfrog 4.520b1 –highnew 767.734 248,696,618 0.465
optimfrog 4.600ex –normal 169.000 254,785,551 0.477
optimfrog 4.600ex –highnew 807.500 248,372,857 0.465
tta 3.3 39.062 265,131,214 0.496
shorten 3.60 59.562 285,314,649 0.534
la 0.4b 498.578 246,284,107 0.461
la 0.4b -high 670.157 244,899,022 0.458
Arsis :: A Celebration of Guilt
Codec Version Mode Encoding Time (s) Size (b) Ratio
wav 0.000 470,087,228 1.000
flac 1.1.2 -5 47.406 357,284,574 0.760
flac 1.1.2 -8 225.313 356,653,905 0.759
flac 1.1.3b2 -5 48.375 355,759,426 0.757
flac 1.1.3b2 -8 226.563 355,026,559 0.755
wavpack 4.31 37.422 357,880,172 0.761
wavpack 4.31 -h 51.515 351,840,948 0.748
wavpack 4.4a3 36.703 354,502,682 0.754
wavpack 4.4a3 -h 46.594 352,533,782 0.750
ape 3.99 -c2000 56.235 349,335,796 0.743
ape 3.99 -c4000 112.828 348,301,840 0.741
ape 4.01b2 -c2000 54.546 349,335,796 0.743
ape 4.01b2 -c4000 114.000 348,301,840 0.741
optimfrog 4.520b1 —normal 156.015 348,831,283 0.742
optimfrog 4.520b1 —highnew 682.797 347,639,982 0.740
optimfrog 4.600ex —normal 151.390 348,831,283 0.742
optimfrog 4.600ex —highnew 714.610 347,208,653 0.739
tta 3.3 38.500 352,991,173 0.751
shorten 3.60 36.578 379,095,497 0.806
la 0.4b 459.031 346,208,027 0.736
la 0.4b -high 617.063 345,982,521 0.736
Jethro Tull :: Thick as a Brick
Codec Version Mode Encoding Time (s) Size (b) Ratio
wav 0.000 462,415,848 1.000
flac 1.1.2 -5 46.500 247,854,895 0.536
flac 1.1.2 -8 212.079 246,862,094 0.534
flac 1.1.3b2 -5 37.672 241,277,363 0.522
flac 1.1.3b2 -8 208.313 238,148,947 0.515
wavpack 4.31 33.469 246,171,638 0.532
wavpack 4.31 -h 46.890 228,449,488 0.494
wavpack 4.4a3 27.953 238,034,152 0.515
wavpack 4.4a3 -h 38.109 231,120,444 0.500
ape 3.99 -c2000 52.109 225,538,128 0.488
ape 3.99 -c4000 110.329 219,826,816 0.475
ape 4.01b2 -c2000 50.278 225,538,128 0.488
ape 4.01b2 -c4000 109.386 219,826,816 0.475
optimfrog 4.520b1 —normal 136.203 221,624,064 0.479
optimfrog 4.520b1 —highnew 665.563 216,749,012 0.469
optimfrog 4.600ex —normal 144.329 221,624,064 0.479
optimfrog 4.600ex —highnew 700.641 215,895,529 0.469
tta 3.3 29.000 234,080,904 0.506
shorten 3.60 23.172 270,694,825 0.585
la 0.4b 434.100 214,990,935 0.465
la 0.4b -high 580.969 213,924,640 0.463
Tool :: Lateralus
Codec Version Mode Encoding Time (s) Size (b) Ratio
wav 0.000 835,642,124 1.000
flac 1.1.2 -5 88.141 486,819,165 0.583
flac 1.1.2 -8 388.375 484,840,473 0.580
flac 1.1.3b2 -5 80.859 481,721,302 0.576
flac 1.1.3b2 -8 390.969 478,689,150 0.573
wavpack 4.31 59.828 482,605,862 0.578
wavpack 4.31 -h 86.485 470,368,632 0.563
wavpack 4.4a3 58.625 477,464,172 0.571
wavpack 4.4a3 -h 76.594 473,126,598 0.566
ape 3.99 -c2000 93.797 464,724,316 0.556
ape 3.99 -c4000 197.657 459,182,396 0.550
ape 4.01b2 -c2000 91.358 464,724,316 0.556
ape 4.01b2 -c4000 195.327 459,182,396 0.550
optimfrog 4.520b1 —normal 260.672 461,868,216 0.553
optimfrog 4.520b1 —highnew 1203.265 456,239,112 0.550
optimfrog 4.600ex —normal 260.594 461,868,216 0.553
optimfrog 4.600ex —highnew 1263.563 455,264,204 0.545
tta 3.3 60.797 474,968,115 0.570
shorten 3.60 98.609 518,718,721 0.621
la 0.4b 782.938 453,892,798 0.543
la 0.4b -high 1051.547 452,745,384 0.541
Havergal Brian :: “Gothic” Symphony (disc 2)
Codec Version Mode Encoding Time (s) Size (b) Ratio
wav 0.000 553,218,668 1.000
flac 1.1.2 -5 49.704 241,800,075 0.437
flac 1.1.2 -8 250.375 240,648,254 0.435
flac 1.1.3b2 -5 49.891 236,724,341 0.428
flac 1.1.3b2 -8 250.156 233,458,574 0.422
wavpack 4.31 31.140 236,816,962 0.428
wavpack 4.31 -h 55.610 228,707,534 0.413
wavpack 4.4a3 38.281 232,951,090 0.421
wavpack 4.4a3 -h 48.485 230,076,294 0.416
ape 3.99 -c2000 61.453 224,760,556 0.406
ape 3.99 -c4000 129.922 218,163,360 0.394
ape 4.01b2 -c2000 60.364 224,760,556 0.406
ape 4.01b2 -c4000 127.653 218,163,360 0.394
optimfrog 4.520b1 —normal 161.516 221,309,324 0.400
optimfrog 4.520b1 —highnew 793.250 215,047,017 0.389
optimfrog 4.600ex —normal 170.640 221,309,324 0.400
optimfrog 4.600ex —highnew 834.813 214,260,553 0.387
tta 3.3 40.328 231,369,533 0.418
shorten 3.60 33.250 251,284,969 0.454
la 0.4b 510.531 214,228,178 0.387
la 0.4b -high 684.906 213,064,890 0.385
Poulenc :: Dialogues des Carmelites
Codec Version Mode Encoding Time (s) Size (b) Ratio
wav 0.000 838,198,748 1.000
flac 1.1.2 -5 77.094 343,708,712 0.410
flac 1.1.2 -8 381.234 342,171,466 0.408
flac 1.1.3b2 -5 76.860 338,312,401 0.404
flac 1.1.3b2 -8 385.360 335,795,645 0.401
wavpack 4.31 54.422 338,463,530 0.404
wavpack 4.31 -h 84.657 330,617,706 0.394
wavpack 4.4a3 56.500 337,399,346 0.403
wavpack 4.4a3 -h 74.797 332,932,224 0.397
ape 3.99 -c2000 94.313 324,081,824 0.387
ape 3.99 -c4000 194.656 314,759,020 0.376
ape 4.01b2 -c2000 92.648 324,081,824 0.387
ape 4.01b2 -c4000 193.231 314,759,020 0.376
optimfrog 4.520b1 —normal 259.234 320,562,281 0.382
optimfrog 4.520b1 —highnew 1198.375 310,232,893 0.370
optimfrog 4.600ex —normal 265.329 320,562,281 0.382
optimfrog 4.600ex —highnew 1261.015 309,379,572 0.369
tta 3.3 57.454 334,248,490 0.399
shorten 3.60 51.578 360,649,861 0.430
la 0.4b 772.844 309,081,279 0.369
la 0.4b -high 1037.719 307,359,296 0.367

Tests performed by Ben Gunnink on November 14-17

§1509 · November 20, 2006 · Tags: , , , , , ·

Leave a Reply