In defense of ABX testing

01-20-15: Zd542

"Back in the medieval days was it science or logic for the times when doctors used to bleed a patient saying the patient had too much blood?"
01-20-15: Jea48

Neither. It was stupidity. They has no way of knowing how much blood was too much blood. The only thing they knew for sure was that if you lost enough blood, you died.
01-20-15: Zd542

LOL, but doctors in that time period of history didn't know any better.

Maybe 25 years from now there will be a machine that will accurately test the differences heard from equipment and cables. Until then we will just have to muddle through using our ears and the processor between them.

LOL, you are asking for a set of rules for ABX testing. I would first start with people who can actually hear differences. I would weed out any individual that is skeptical of an ABX listening test. Next I would probably want the participant's hearing tested by an ENT doctor in a hearing test booth. Those that do poorly they are weeded out. After the participants are finally selected only one person at a time would enter the listening room. The test could be conducted just like a hearing test at the ENT doctor’s office. The first Set of ABX hearing tests are strictly if the person can hear a difference from A&B or no differences from A&A or B&B. For those that can accurately hear the differences then those listeners would move on to the next level what they perceive sounds better to their ears.
.

Jea48, I think if they did it the way you recommend, then we'd have nothing to argue about. :-)

One of the difficulties in behavioral research is precise definition of stimulus-response. The more complex the stimulus, the less precise the definition. In ABX testing, the definition of the cumulative response is trivial: can reliable hear a difference or not. But because the stimulus is likely to be imprecisely defined, absence of reliably making the distinction does not mean there is no difference. For music, Gestalt seems too relevant. That's one of the reasons we know so much about what a rat is likely to do in a maze and so little about what a kid is likely to do in a classroom.

db

Another being a kid is not a rat.

"LOL, but doctors in that time period of history didn't know any better."

That was my point. The only thing they knew for sure about blood was that if you loose too much of it, you die. Armed with only that info, I stand by my statement. lol. Maybe there was some logic to it, but I don't see. I guess its possible that every time a doctor saw a bleeding, injured person, he thought that was the body getting rid of some excess blood. Why blame it on the arrow stuck in the arm.

About the rest of your post, my comments on all this, in context, is in reference to the thread the OP mentioned on Hydrogen Audio. The complaint was that reviewers were listening to audio components and then basing their review on what they heard, and not doing any type of scientific listening tests. They've complaining about this very thing for years. My comment was that if these types of test are so damn important, then just do them already. Even if its just a few tests just to show us all how to do it. Instead, its just year after year complaining, and they never do anything. That said, if they can come up with some kind of useful tool to help better evaluate audio equipment, I know I would be interested in seeing it. Why not? My personal opinion is that they don't have the guts to do anything they talk about. There's always the chance they would be wrong. They have too much invested in the argument.

"In ABX testing, the definition of the cumulative response is trivial: can reliable hear a difference or not. But because the stimulus is likely to be imprecisely defined, absence of reliably making the distinction does not mean there is no difference."

I agree with that. My view is that you would have to tell the test subjects what they're listening for. There's really no way around it, they have to know. To offset results that are not accurate, you can increase the number of tests, or try's, each subject takes. So, for example, if you were trying to test to see if a difference can be heard between a silver cable and a copper cable, all other things being equal, maybe have them listen to 50 or 100 samples. Maybe they can get lucky and guess correctly for 5 or 10, but 100 is highly unlikely. Not only that, under the same scenario, you can tell them exactly what they should be listening for. If there is really no difference to be heard, over time/individual tries, the test subjects will have to trend towards a 50/50 split. It won't matter what they think, or know they can hear.