A Modest Construct

Posts tagged `FLAC`

¶ Lossless Audio Shootout!

On the heels of my recent comparison of general-purpose data compressors, I bring you the results of far too much time spent hunched in front of a command line, calculating ratios.

The benchmark’s ostensible purpose, at least at first, was to compare four mainstream codecs—FLAC, Monkey’s Audio, WavPack, and OptimFROG—and see how their current stable versions stacked up with the version in development. I ended up throwing in a few others—TTA, LA, and Shorten—for comparative purposes. This test looked at Encoding Time and Compression Ratio; decoding speed was not tested.

If you don’t want to look at the raw data, I’ll save you the trouble of even clicking past the fold.

FLAC: FLAC has languished at v1.1.2 since February of 2005. It has the uncomfortable position of having reached an stable position in the world of lossless codecs, even garnering some hardware support. As a result, it can’t make any sweeping changes without breaking compatibility or hurting its (excellent) decoding speed.; The result is that there have been some definite improvements in compression, though nothing that will blow anyone away. There appears to be an album-wide net loss of between 0.5 and 1.5%, enough to shave another few megabytes from the total size. The default compression (-5) for 1.1.3b2 now compresses better than the maximum practical compression (-8) of 1.1.2. Of course, part of 1.1.3′s appeal is other features, like album art embedding, but that’s beyond the scope of my test.
WavPack: WavPack has been around for a long time (it supposedly inspired Matt Ashland, the creator of Monkey’s Audio, to work on his own), but for some reason has always been underappreciated in the world of lossless compressors. I suppose that’s because it manages to fall somewhere in the middle of Monkey’s Audio and FLAC, not offering compression ratios as low as the former or decode speeds as high as the latter. Still, I was impressed that WavPack not only encoded faster than FLAC, but it got a better compression ratio, too—and it’s decoding speed is also very excellent.; WavPack has two settings: default and -h, or high. It also has a -x[1-6] switch that can possibly shave off a fraction of the size at the expense of a really long encode time. The tendency of 4.4a3 under the default setting was to take a few seconds longer to encode with the benefit of between 0.1% and 1.5% improvement in the compression. For the high setting, however, the encoding time was a few seconds quicker, but the ratio was always worse.
Monkeys Audio (APE): Monkey’s Audio has a storied history, part of which has to do with its laughable “open source” license, spurned by every Linux distribution I’ve ever seen. APE offers better compression than FLAC without drastically increased encoding times, but it’s particularly CPU-intensive for decoding, making it impractical for handhelds or DAPs.; The change from 3.99 to 4.01b2 has been entirely in the GUI frontend. The encoding times weren’t different enough to be significant, and the sizes were exactly the same. My understanding, though, is that it’s been optimized for dual-core processors, so if you’ve got one, you’ll see a significantly faster encode time.
OptimFROG: OptimFROG is the only truly closed-source encoder in the primary four. It offers the best compression of them as well, but also long encode times and piss-poor playback speeds. It hasn’t gained very much traction except as a curiosity: its closed source nature prevents its use in Linux distros, and its intense resource usage means it will likely never be supported in hardware.; The changes in OptimFROG between 4.520b1 (stable) and 4.600ex (testing) aren’t in the default encoding options. The —highnew mode (the —bestnew mode took 100 minutes for the first album, so I very quickly decided to try the one below it), which increased encoding time for modest compression gains under 0.5%.
Others: TrueAudio is another open-source format that’s languished in obscurity. It’s resolutely middle-of-the-road, offering decent compression and a fast speed. Unfortunately, that means it also gets shown up by the Best in Class codecs.; Shorten was the very first of the serious lossless encoders, first showing up in 1993. It hasn’t gone anywhere in years, and even the organization most attached it to (the live music trading site, ETree, long ago replaced it with FLAC. It offers super-fast decoding speed, but the very worst compression.; LA is another closed-source encoder with very slow encoding times (not quite as bad as OptimFROG) that repeatedly trounces every other codec available in terms of compression ratio. However, its high resource usage and closed nature make it impractical for just about everything but a curio. It also hasn’t seen any development in some time now.

To see the complete table, look below the fold.

¶ Go Western, young man!

Disclosure • This blog entry deals with a lot of people and a lot of places. Although nothing I could divulge has any reasonable expectation of privacy, I am still hesitant to include too much identifying information about parties who may not wish to be associated with this blog or its content. I am largely eschewing the use of any names beyond mine and Allison’s, including any relevant fraternities or sororities.

The Rising Action

I spent the past weekend in Macomb, Illinois, a little bucolic ‘burb in the middle of a vast stretch of corn and grass. In any other circumstance, this sleepy little town of about 20’000 people would be entirely unexceptional—it is quintessentially Midwestern, by which I mean that it is quintessentially dull. Officially founded in 1830, it didn’t gain its famous feature—Western Illinois University—until 1899. It enjoyed a brief relationship with the St. Louis Rams, who used WIU’s athletic facilities for summer training from 1996-2004.

I made the 3-hour drive west to see my girlfriend, Allison, who began her undergraduate studies there this fall, as you may recall if you read this rag regularly. We’d planned it for close to a month—I knew that visiting was a categorical imperative for me, but it was just difficult enough to remain an occasional thing. Sadly, no surprise visits or unexpected trips. Read more…

§1381 · September 27, 2006 · 2 comments · Tags: Allison, FLAC

¶ Terrorists < Pirates

Ed at Freedom to Tinker comments on the triennial DCMA exception discussion.

Here’s a basic overview of the problem. Opponents of DRM say:

There should be a stipulation allowing “users to remove from their computers certain DRM software that causes security and privacy harm”
Exemptions to bad DRM would mean that labels would be under presssure to come up with good DRM
Or, as worded by the CCIA and Open Source and Industry Association: add an exemption for DRM schemes that “employ access control measures which threaten critical infrastructure and potentially endanger lives.”

Then Big Media (BSA, RIAA, MPAA, &tc.) says:

There wouldn’t be any incentive to create better DRM it “would be fundamentally undermined if copyright owners [...] were left in such serious doubt about which measures were or were not subject to circumvention under the exemption.
[T]he boundaries of the proposed exemption would turn on whether access controls “threaten critical infrastructure and potentially endanger lives”

Now, I agree with Big Media insofar as the proposed exemptions are rather broadly defined, and you know that Big Media hates broad language unless it works to their benefit. However, when we’re talking about what defines a situation where DRM “threaten[s] critical infrastructure and potentially endanger[s] lives,” something tells me that we’re not talking about Joe Sixpack illegally obtaining the latest Korn album. That Big Media would quibble over such things is truly pathetic.

I’m surprised that they’re even pushing DRM so hard at all. Anyone who knows anything about technology has been saying for years that it’s a waste of time. It ruins your portable music player’s battery life, it can harm your computer, it doesn’t accurate reflect the usage pattern of most consumers, it’s not broadly compatible, and it hasn’t seemed to make a dent in the RIAA/MPAA’s bogeyman of INTERNET PIRACY!

Here’s the thing: if my options are to pay $X to download a 128kbps AAC, MP3, or WMA file with copy protection that I can’t use indefinitely or play everywhere, or to “illegally” download a release group’s rip at 192kbps/VBR or find a lossless FLAC version somewhere, then my choice is obvious. I hate to break it to big media, but as much as they’d like to be the ultimate arbiters of price, technology, and taste, it’s really market forces. I’m part of that market, I cast my vote, and my vote is a big hearty “Fuck you” to the RIAA, the MPAA, software patent holders, and anybody else who thinks they deserve money for controlling information instead of providing services.

§1051 · March 22, 2006 · (No comments) · Tags: FLAC, open source, politics, software, technology

¶ A Declaration of Audio Rights

1. Patents

There should be no reason whatsoever that patent-free formats (OGG, FLAC, APE, MPC) shouldn’t be supported by every audio program and digital audio player in existence. The code is freely available: the only excuse for companies who continue to churn out hardware players that only support MP3 and WMA is sheer laziness. We know that it’s possible. Even vendors who like to tout themselves as friendly to FOSS continue to enforce a narrow spectrum of patented codecs (MP3 and WMA or AAC) and vendor lock-in. From now on, consumers must press for the adoption of patent-free codecs for their own music libraries, and in terms of vendor support.

2. Digital Restrictions Management

DRM is a fundamental blow to users’ freedom of information. DRM is only supported by proprietary audio codecs whose performance and quality is severely lacking, and only understood by restrictive software like Windows Media Player and iTunes. It hinders the fair use of legally purchased music and does nothing to hinder the spread of pirated material, all of which is available in formats with freely available encoders (FLAC, LAME, &c.). It does this by only allowing a song to be transfered a certain number of times before expiring, meaning that despite legally purchasing the song from whatever source, the consumer does not own the music, but is rather only renting it; all this for a song in a poor format, a fraction of its former self. From now on, consumers must let vendors know that DRM is an abridgement of their rights as consumers and an insult to free ownership.

3. Bitrates

Vendors tout the size of their Digital Audio Players’ hard drives (which might otherwise be incomprehensible to computer-illiterate customers) in terms of the number of songs that can be crammed onto it. In order to maximize the appeal, marketing departments usually use the lowest possible encoding quality in order to use bigger numbers. The Creative Zen description states “20 GB storage lets you bring up to 10,000 WMA songs (64 kbps) or 5000 MP3 songs (128 kbps) everywhere you go.” It is in affront to audiophiles everywhere to suggest that one should encode anything at 128kbps; such a thing may have been acceptable in the days of Napster, but is certainly not now. MP3 files do not reach transparency until at least 192kbps. It is a mockery of even greater proportions to suggest that any piece of music should ever be subject to an encoding at 64kbps (in WMA, even!).

It is time that consumers start demanding a more realistic approach to these marketing campaigns; they need better education on the relative qualities of various formats at differing levels of compression. What’s more, disk-based storage has reached an extent at which we should be clamoring for better (if less efficient) methods of compression, such as the aforementioned lossless codecs. From now on, consumers must pressure manufacturers to use realistic figures when describing capacities.

4. Bono

Bono must stop making commercials for the iPod and just focus wearing sunglasses at diplomatic functions. In fact, he should probably stop making albums as well; he seems to be better as an activist than a musician.

5. Radio

Consumers need to abandon the radio as a means of entertainment. It’s become a cesspool of nothing but Top 40 tripe and the vapid banter of legally retarded show hosts. What’s more, its quality is completely miserable, and it’s subject to the iron fist of the FCC. Either listen to CDs or MP3 players in your car/home, or use satellite radio as a censor-free and higher-quality alternative. The idea of listening to six minutes of commercials in order to hear the same Gwen Stefani song they played an hour ago strikes any sensible audiophile as absurd. Take control of what you listen you.

6. Mastering

As stated before, the tendency to create extremely loud songs suited to radio airplay is a death knell for music quality. To all producers and sound engineers, this is a call to start mixing audio in such a way that preserves dynamics and makes the experience of listening to said music more enjoyable. Not only does hot music just sound bad, but it makes the job of compressing and manipulating that music even harder. Stop it! Just stop it! Ross Robinson, I’m talking to you!

7. Consumers

Of course, none of this would be a problem if you music fans and consumers weren’t such braindead twits about it all. You have the right to inform yourselves of the dangers of DRM, the strengths and weaknesses of different compression formats, and the evil that is hot mastering. You have the right to pester hardware vendors about support for patent-free codecs and the right to purchase from those companies that do implement them; you have the right to see through marketing bullshit about fitting 10’000 songs on a single player. You have the right to not give a damn about Apple’s glossy-but-inferior products or the white earbuds that will get you mugged. You have the right to use better programs than Windows Media Player, the right to encode your entire collection in FLAC, the right to transfer a song that you own to as many devices as you damn well please.

You have the right not to be a lazy as and accept whatever is fed to you. And you have the right to enjoy your music in all its unadultered glory.

§680 · July 12, 2005 · 12 comments · Tags: FLAC, music, software, technology

¶ The loudness race and audio compression

The Loudness Race: the aim of pop music producers to create sounds that are bigger and louder in order to sound “better” on the radio. It’s a sickness, really. Abou talked about it almost a year ago, though unfortunately the site he links to is gone (though you can sort of view it with the Wayback Machine).

One of the major problems that have cropped up in many of the pop music releases in the past 5 to 6 years is level control. I have painfully listened to my step kids CDs and heard the shrilling sounds they conveyed. I wasn’t surprised when I hooked up the CD player’s analog outputs to my oscilloscope and observed levels so high that they were clipping on a consistent basis. One normally thinks of clipping as a power amplifier problem, yet clipping in the digital domain can be just as bad.

I bring this up, because lately I’ve been experimenting with the FLAC audio compression format. It’s touted as having an average compression ratio of 0.5-0.55 (original size * 0.5[5] = new size). When I was converting a CD of Rachmaninov piano concertos, I got between 0.38 and 0.47. Obviously, those are extremely good compression ratios for a lossless encoder. In fact, they are such that I’ve considered buying some extra storage space and having my entire music collection be FLAC. Except I can’t. Because most of my collection is modern, and probably hot as hell. I tried encoding my Boy Hits Car album in FLAC, and got average ratios of 0.75 (that’s bad: only a quarter of the file size was shaved off). At 400 modern CDs, that’s almost 300GB right there, not counting anything else I download.

And yet, when I encoded King Crimson’s In the Court of the Crimson King, I was getting the promised levels of between 0.5 and 0.6. Why? Because that album was recorded and mastered well, just like most recordings of classical music. With the proper levels, FLAC is able to achieve good compression rates. Another album that is supposed to be excellent in regards to this is Tool’s Lateralus according to Wes Lindstrom. In an effort to test my hypothesis about FLAC, I decided to do a rough comparison of rates for a robust classical album (Solti/CSO play Mahler’s 8^th Symphony), a relatively tame classical album (Ashkenazy’s performance of some Rachmaninov preludes), Tool’s Lateralus, Opeth’s Blackwater Park, RHCP’s Californication, Oasis’s (What’s the Story) Morning Glory?, Änglagård’s Epilog, and Jethro Tull’s Thick as a Brick. I will average the ratios for all tracks and present the findings, seeing if the compression matched what I thought it would be based on my preconceptions of the mastering. For instance, I expect both classical albums to be excellent, as well the Jethro Tull and to a lesser extent the Änglagård. Opeth will fall in the middle, and Oasis/RHCP will be horrible. All compression is done using FLAC 1.1.2a -8 -V.

Album	No. of tracks	Mean ratio	Whole album ratio
Opeth • Blackwater Park	8	0.680	0.714
Oasis • {What’s the Story} Morning Glory?	12	0.684	0.689
Red Hot Chili Peppers • Californication	15	0.707	0.708
Jethro Tull • Thick as a Brick	2	0.534	0.534
Änglagård • Epilog	6	0.439	0.520
Tool • Lateralus	13	0.540	0.580
Gustav Mahler • Symphony Nr. 8 (Solti/CSO)	16	0.518	0.499
Sergei Rachmaninov • 24 Preludes [Disc 1] (Ashkenazy)	15	0.342	0.337

As you can see from the results, most of my hypothesis was confirmed. Classical music, even boisterous Mahler, compresses extremely well, while modern music does not. Depending on the type of music, a similar approach may or may not be taken in terms of mastering and levels. The Änglagård album, recorded in the the style of 60s prog bands, saw compression rates similar to those touted by FLAC’s coders. Californication, mentioned specifically by Wes Lindstrom for being extremely hot, sees levels at about 0.75. That was 1999: today, most rock/pop albums are that hot. Lindstrom also specifically mentioned Tool’s Lateralus as being well-mastered, and we can see that the main tracks tend to fall in the low 0.6s: not exactly the best I’ve ever seen, but excellent for an album with as many furious moments as it has. Even Mahler’s 8th hit similar numbers at one point.

I’m no expert on the acoustic model, but it doesn’t take a neurosurgeon to figure out that the music industry’s tendency to create extremely loud/hot albums (think of these in terms of the acoustic model) is not only damaging to the quality of the music, but it also defies our ability to compress it efficaciously. When you consider that pop bands essentially record for the radio, it’s really no surprise that most people can’t appreciate the difference between a well-mastered and a poorly-mastered album. It’s also no wonder that people don’t seem to mind listening to mp3s with such low bitrates: these songs were recorded for a lackluster medium, and one lackluster medium is as good as another. Hot mastering steals away dynamic contrast and quality sound. Take heed, producers: give us back our music!

§645 · June 14, 2005 · 8 comments · Tags: FLAC, music, technology