2. tech-talk
  3. 1421262840

In defense of ABX testing

We Audiophiles need to get ourselves out of the stoneage, reject mythology, and say goodbye to superstition. Especially the reviewers, who do us a disservice by endlessly writing articles claiming the latest tweak or gadget revolutionized the sound of their system. Likewise, any reviewer who claims that ABX testing is not applicable to high end audio needs to find a new career path. Like anything, there is a right way and many wrong ways. Hail Science!

Here's an interesting thread on the hydrogenaudio website:

http://www.hydrogenaud.io/forums/index.php?showtopic=108062

This caught my eye in particular:

"The problem with sighted evaluations is very visible in consumer high end audio, where all sorts of very poorly trained listeners claim that they have heard differences that, in technical terms are impossibly small or non existent.

The corresponding problem is that blind tests deal with this problem of false positives very effectively, but can easily produce false negatives."
psag
nonoise's avatar
nonoise
4,382 posts
 
This dead horse has been flogged for so long that it's turning me into a vegetarian.
Post removed 
zd542
5,339 posts
 
"01-17-15: Bob_reynolds
Drs. Floyd Toole and Sean Olive have been doing blind listening tests of loudspeakers for over a decade."

I've seen all these before. I'm assuming everyone else here has too because they are fairly popular. They don't address the issue here, and that's comparing specific pieces of components. Here's a piece of the OP.

"Especially the reviewers, who do us a disservice by endlessly writing articles claiming the latest tweak or gadget revolutionized the sound of their system."

Can you show us some scientificly valid listening tests that were done comparing individual components as part of a review?
geoffkait's avatar
geoffkait
14,147 posts
 
Judging from the AES paper by Olive the listening tests are, in fact, excessively complicated, a criticism he dismisses. Furthermore the listening tests apparently involved only frequency response. What happened to other audiophile parameters such as musicality, transparency, soundstaging ability, dynamics, sweetness, warmth, micro dynamics, pace, rhythm, coherence, to name a few? One supposes testing for those parameters would make the tests way too complicated. Maybe Olive thinks those parameters are too subjective, who knows?
geoffkait's avatar
geoffkait
14,147 posts
 
Scientifically valid is the key phrase. There is no agreement in the scientific community with respect to audio tests. In fact, the scientific community could give a rat's behind about audio or audiophiles or testing audiophile devices, any of that. Hel-looo! If someone says he represents the scientific community in any of this controlled blind testing business or any type of testing for that matter, he's just pulling your leg.
More to discover