In defense of ABX testing


We Audiophiles need to get ourselves out of the stoneage, reject mythology, and say goodbye to superstition. Especially the reviewers, who do us a disservice by endlessly writing articles claiming the latest tweak or gadget revolutionized the sound of their system. Likewise, any reviewer who claims that ABX testing is not applicable to high end audio needs to find a new career path. Like anything, there is a right way and many wrong ways. Hail Science!

Here's an interesting thread on the hydrogenaudio website:

http://www.hydrogenaud.io/forums/index.php?showtopic=108062

This caught my eye in particular:

"The problem with sighted evaluations is very visible in consumer high end audio, where all sorts of very poorly trained listeners claim that they have heard differences that, in technical terms are impossibly small or non existent.

The corresponding problem is that blind tests deal with this problem of false positives very effectively, but can easily produce false negatives."
psag
Its a tool. No tool is perfect. If it fits the task at hand use it. Just don't expect anyone else will draw the same conclusion. They may or may not and it should not matter other than as a point of interest. Only you can hear what you hear. No one else. Being of the same species, we all hear similarly perhaps but not exactly the same. Some differences may be major others subtle. The subtle ones will probably never be measured or quantified so just forget about it.
01-20-15: Zd542

"Back in the medieval days was it science or logic for the times when doctors used to bleed a patient saying the patient had too much blood?"
01-20-15: Jea48

Neither. It was stupidity. They has no way of knowing how much blood was too much blood. The only thing they knew for sure was that if you lost enough blood, you died.
01-20-15: Zd542

LOL, but doctors in that time period of history didn't know any better.

Maybe 25 years from now there will be a machine that will accurately test the differences heard from equipment and cables. Until then we will just have to muddle through using our ears and the processor between them.

LOL, you are asking for a set of rules for ABX testing. I would first start with people who can actually hear differences. I would weed out any individual that is skeptical of an ABX listening test. Next I would probably want the participant's hearing tested by an ENT doctor in a hearing test booth. Those that do poorly they are weeded out. After the participants are finally selected only one person at a time would enter the listening room. The test could be conducted just like a hearing test at the ENT doctor’s office. The first Set of ABX hearing tests are strictly if the person can hear a difference from A&B or no differences from A&A or B&B. For those that can accurately hear the differences then those listeners would move on to the next level what they perceive sounds better to their ears.
.
Jea48, I think if they did it the way you recommend, then we'd have nothing to argue about. :-)
One of the difficulties in behavioral research is precise definition of stimulus-response. The more complex the stimulus, the less precise the definition. In ABX testing, the definition of the cumulative response is trivial: can reliable hear a difference or not. But because the stimulus is likely to be imprecisely defined, absence of reliably making the distinction does not mean there is no difference. For music, Gestalt seems too relevant. That's one of the reasons we know so much about what a rat is likely to do in a maze and so little about what a kid is likely to do in a classroom.

db