In defense of ABX testing

Personally I have noticed that because in a full track of audio - there is so much information - it is easy to become overwhelmed with all the information and miss the subtle differences. One person explained to me that it can be similar if you look down a microscope - if you focus your attention on an aspect of one part of the audio - you can miss the rest. I have found the solution to this is to listen many times to the testing audio - in order to get used to it. My results improve dramatically in this case-going from failure - to 10/10. I suspect many people do not - which could lead to them getting worse results.

Also - I find I need to loop very short segments of audio - and I understand this is because of the brain cannot remember for very long audio excerpts (?)...Im talking around 2-3 second clips.

What about with music - which is something people often 'feel' subconsciously - when put into a position where they consciously evaluate - are they able to translate the subconscious feeling into a concept they can then consciously correctly identify? There are considerations about how our brain work that may impact the results also I think...

Quote from Link

I see no problem with what the guy is saying.
.

"Nobody is denying the existence of placebo effect or expectation bias or it's ugly sibling the reverse expectation bias or any other such psychological effects. But to declare that there are no proper tests is a little bit inaccurate."

No, its not. If you take my quote that you reference, "For years I've been asking them to show me the tests.", and put it in context with the rest of my statement:

"If you're right, and you really know what you are talking about, pick 2 products, conduct the test, and report your findings in a way so the rest of us can try it for ourselves. That's how a real scientist would do it."

we get a different picture. To me, if you wanted to conduct a scientificly valid listening test to compare 2 audio products, I can't see you going wrong doing it the above way. Not only that, if you read through the thread that the OP referenced, you'll see that, at least some of them, do agree with me. So again, it all boils down to the same exact thing. Show me the test's. If me declaring that there are no proper tests done is a little bit inaccurate, by all means, show me. I'll settle for just 1. And just to be clear, I really have been asking for years. I'm 100% serious. If you really want to, you can check my old AG threads.

Sorry, but before I forget; "I suspect maybe you've been asking the wrong people.". Maybe you're right about that. Who do I ask? I mean if I don't get results at a place like Hydrogen, where its their mission statement to go by science, and will actually censor threads if they don't contain content approved by the moderators, then who do I ask? If you really take a step back and look at this whole issue, the people who talk about doing these listening tests the most, avoid it like the plague. It doesn't even make sense.

I agree with the notion of Science! being applied as a tool for understanding how different equipment influences my perceptions, but I'm not convinced ABX testing is the right way to go about it.

Even if a test population of reasonable size is acquired, large enough to smooth out the differences in loudness perception in the subjects, you'd still only have a result for that particular set of equipment. Perhaps a statistically significant percentage of the subject group could identify MP3 versus APE, but once you swap out a cable or change the room temperature, confidence in the results degrade.

If one wants to apply science here, there first has to be a hypothesis that can be falsified through measured experiment. For example, "all conducting wire measures the exact same frequency response curve with white noise at 90db, regardless of the transducer". This is probably trivial to falsify, but is on the same continuum of the notion that high priced cables are "better" than lamp cord.

If you want to apply science to that or a similar or even more refined question, super. But an ABX test isn't going to get you a definitive result.

This dead horse has been flogged for so long that it's turning me into a vegetarian.