Blind Audio Testing: The Scientific Truth Behind the Myth

4
4774

Audiophiles frequently “discuss” blind testing.  In some people’s opinion, doing a “blind test” is the only way to test whether or not one audio component sounds different than another.  “All amplifiers sound the same. It’s been proven in blind tests!”  For others, the validity of blind testing can be problematic because of a variety of different variables.

Today, I was listening to the Home Theater Geeks podcast with Stereophile’s John Iverson.  (As an aside, this is a truly excellent weekly segment hosted by Scott Wilkinson.  I recommend you check it out if you haven’t).  John Iverson made a point in the interview about blind testing that I wanted to highlight because I think he’s spot on.  Let me explain.

Blind testing isn’t so much about the differences in the equipment as much as it is a test about the listener.  Moreover, just because a listener is able to perceive a difference between Product A and Product B, you can only talk about the listener’s preference for a particular product.  If you want to take the variable of the listener out of the equation, then your listener panel needs training.

The importance of the listener in any kind of audio test simply cannot be emphasized enough in my opinion.  As someone who has taken some listening training I can say from experience that this is a wonderfully humbling exercise.  In fact, I will even go so far as to say that using untrained listeners in a “blind” test is a variable that should be explicitly stated in any test.  If you’re skeptical of those claims, then I suggest you check out the benefits of using trained listeners as documented here:

John Iverson’s comments are right in line with the work pioneered by Dr. Floyd Toole and now Dr. Sean Olive of Harman International.   Harman is the parent company of the luxury audiophile brands Mark Levinson, Revel, JBL and Lexicon.  I’ll highlight for a moment some previous work posted by Dr. Olive from his blog post here about a “Method for Training Listeners and Selecting Program Material for Listening Tests”.  I’d like to point out that when Harman’s own Revel speakers subsidiary tests their speakers, they only use trained listeners.

There are two programs that I know of that the average enthusiast can participate in to become a better, trained listener.

The first is Harman’s own “How to Listen” software.  It’s a free download for Mac and Windows computers.  It comes complete with sound clips that will help train you on becoming a trained and better listener.  The software works by presenting you with an audio sample.

Harman’s How to Listen software is a free download for Mac and Windows computers

It will then introduce some variable into the audio sample.  You then need to then specifically identify what that variable is.  You’ll be able to track your progress and pass from one level to another once you’ve “mastered” that particular section.

The software is incredibly flexible.  It will give you the options to input your speaker setup and fine-tune several aspects of the program material.

 

The second is Philips “Golden Ears Training“.  The Golden Ears program.  According to Philips, the Golden Ears training program was designed to develop the listening skills of their audio engineers.

 

Philips “How to Listen” is a free online course you can take to improve your listening skills along key metrics.

Training audio engineers was a way Philips felt it could create products of superior sound quality that highlighted every musical detail.  Philips states that graduates of the Golden Ears program can reliably detect subtle differences in sound in timbre, details, spacial impression, bass, and loudness/dynamics.  Users can signup for free and take the Golden Ears challenge and chart their progress accordingly.  While I’ve been part of the Harman program, I haven’t yet tried out the Philips program.

Philips online Golden Ears tutorial site helps you setup your system with a simple, step-by-step walkthrough.

Both are free.  I applaud and tip my hat to both Harman and Philips for making the products free.  Helping people become better listeners is an invaluable contribution to the audio and home theater community.  On behalf of all audiophiles, “Thank you!”

I also want to thank John Iverson for a great interview and great insights.  I also don’t take John’s comments to an extreme by saying that only trained listeners can evaluate equipment.

The next time someone makes a definitive, vocal claim in some Internet forum that “Blind tests have shown that…” or,  “You can’t say that without blind tests…” you can tell them that without trained listeners the only thing you’ve proven in the blind test is something about the listeners, not the equipment being tested.  Oh, and if they question you, just point them to the peer-reviewed research from the major audio industry publications 😉

4 COMMENTS

  1. I don't really understand… You link to several publications about comparisons between speakers discussing the influence of a trained ear. Speakers are, however, one of the few parts of an audio setup that CAN be reliably distinguished in ABX tests, in fact even by amateur listeners, and therefore definitely warrant some research and investment.

    The meat of the blind testing argument is in outrageously priced components like power/audio/HDMI cables which are simply impossible to distinguish from cheaper counterparts, even for trained listeners.

  2. @Sigma, you're certainly correct that speakers are one of those components where you can discern differences. The issue with trained listeners goes one step beyond discerning differences it's identifying what those differences are and being able to accurately describe them and then apply ones preferences to that. Trained listeners are incredibly valuable when it comes to speaker design and testing. Floyd Toole and the team at Harman have done extensive and pioneering research on this area and have published several papers. Floyd's book "Sound Reproduction: the Acoustics and Psychoacoustics of Loudspeakers and Rooms" is one of the classic texts on the topic.

  3. You put the quote marks in the wrong place in the first sentence. It should have been: Audiophiles frequently "discuss" blind testing.

Comments are closed.