Why is Double Blind Testing Controversial?

Tobias and Redkiwi hit it on the head. It takes prolonged exposure (post burn-in, which a lot of A/B's may not do) to really absorb the differences. "Night and day differences" aside, most A/B's are a matter of trade-offs -- e.g., a little more bass punch vs. a little less definition. What seems very nice in the short term may not be preferred longer term. What seems like a strength listening to one recording may not seem so while listening to another. In reality, all of our choices are A/B. We are replacing what we have with something else we believe is "better" after prolonged listening. That's why companies offer 30 day trials rather than 30 minute trials. From my understanding, most A/B tests tend to be 3 minute trials. Sorry, I'm just not that good.

By the way, none of this is to say that I don't harbor a secret fear that I'd pick Bose ;-) But, I have heard too many say that, with prolonged exposure, they can pick out component signatures from a distance without seeing the actual unit/cable being used.

Well, you don't need to use the blind test to make your decision. The idea is to isolate, at least for the period of the test, the contribution of knowing who the manufacturer is, what the product costs, what it looks like, and so forth. Getting those variables out of the way at some point during your evaluation, even briefly, might be helpful, don't you think? Doesn't mean you won't choose the higher priced product in the end, but at least you will have the benefit of some calibration between what you hear and what you perhaps expect or hope to hear.

We've covered this before. As such, all i'll add to this thread is that Craig and Albert cracked me up. If DBT threads are going to be good for laughs like that, bring them on : ) Sean
>

The use of blind and double-blind procedures presumes one is employing the logic of hypothesis testing. That is, that there is a null hypothesis (i.e., that there are no differences between two treatments—in this case, two sets of interconnects) and an alternate hypothesis (there is indeed a difference). Experimenters are more than experimental custodians. Their biases and expectations can profoundly influence a study. To the extent that all people (including experimenters) have biases, one would double-blind the treatments to reduce among other things "experimenter effects." It’s surprisingly easy for an experimenter to influence a study (e.g., Stanley Milgram’s famous obedience studies). It is also easy for other participants (formerly known as “subjects”) to influence each other (e.g., Ash’s line judgment experiments where participants tended to agree with Ash’s confederates that clearly dissimilar lines were the same).

There is a famous researcher/psychologist/statistician by the name of Robert Rosenthal who once told his students that he had obtained two breeds of rats from another famous researcher. One type of rat was called “maze smart” and the other was “maze dull.” Dr. Rosenthal asked the students to teach these rats to run though mazes (ah, the power of cheese). After a few weeks or so the students were asked to show off their rats’ maze prowess (as it were). The “maze smart” rats performed significantly better than their “dumb” counterparts. The kicker here is that the rats were OF THE SAME SPECIES. One cannot infer that the students intentionally influenced the training, but it most certainly was measurable. Moreover, when the experimenter bias was measured it turned out that the “smart" rats owners had "imparted" a greater positive measurement bias than did the “dumb” rats owners negative measurement bias.

There are probably much better examples than these, but I’m in a hurry to go downstairs for dinner :-) so I’ll wrap this up soon.

Something else to consider is that “different” does not mean “better.” People’s ability to remember sounds and colors varies greatly but rarely is the memory accurate after a short decay period. With audio equipment evaluation, it tends to result in a bias for a certain “sound” regardless of whether or not that sound is authentic. When it comes to making a decision as to whether one component is better than another, it probably makes the most sense to have a reference. In the case of audio, I’d say that reference should be THE REAL THING. It’s not practical to have live orchestra tag along on equipment tests but it doesn’t hurt to keep that in mind. Some people go on and on about how they prefer one cable to another because their favorite is “warm” or whatever. Real sounds from an orchestra or a band are not necessarily “warm.”

All that said, if one believes that a $6,000 set of interconnects sounds better (they just might sound *different*) than a $70 pair then let ‘em. The more expensive cable might even sound closer to reality. One would hope that the more expensive cables aren’t just mostly cosmetics and markup.

--Paul

p.s. and yes, spending time with a set of cables or anything else in the system is a great way to know if one really likes the sound. On a marginally related note, a friend of mine once said “I’ve never owed a handheld device that I liked after having it for a week.”

Well said Bomarc. I'll go a little further along that road. Most people who argue in favor of DBT's in these threads imply that audiophiles don't want to discover that they can't tell one thing from another. I think audiophiles believe that DBT's are not good because that's what they've been told by the buff mag writers (aka "reviewers," but really just story tellers), who don't want to be exposed. You see, Sean, in a DBT, Mikey can't tell those preamps apart either.

There are reviewers with real scientific credentials and experience who advocate and use DBT's, but not many.

They don't have to be short. You can listen to whole pieces of music. And you can start by familiarizing yourself with the components under test until you are sure of what it is that makes them sound different, then try to tell them apart blind, not trying to prove there is no difference, but to confirm your hypothesis that they are different. In home, one obvious difficulty is exact matching of spls.