Why is Double Blind Testing Controversial?

TWL, I'm not following your logic or maybe I just didn't read it carefully enough.

First, you make a point that DBT proved to be virtually the same as non-DBT for you. (BTW, congrats on the ~ 100% results. I wish more "reviewers" would do the same exercise.)

Then, you basically claim that DBT is primarily for people that want to save money (or claim that their lower priced system is just as good as any other.) This seems to be a stretch to me.

First, I doubt that most DBT proponents advocate it to justify their inferior system. You're taking a cynical view of an opposing viewpoint and making a generalization. Regardless, even for those who use DBT to solely justify a lower priced product, what other method would you recommend that is better? You, yourself, said the results were identical.

Bottom line -- DBT is either an accurate test or not. If it isn't, then your 100% non-DBT concurrence with DBT tests would be a bad thing wouldn't it???

I advocate this testing as a way of attempting to control variables in subjective evaluations, not as means of disproving the merits of high-end, high- priced components.

Labtec, just because I have used blind testing in the past, does not mean that I consider it terribly useful. I can much more easily get the answer I need regarding the performance of a product, by just doing a simple listening test, and dispensing with all the blindfolds, and mystique of blind testing. It is simply not necessary for me. If I find that something sounds so close to what I have, that I can't really tell, I don't really need to be considering an upgrade to that product. Very simple really.

As far a blind testing being used by others, I don't think that it is used for testing the sound of equipment by very many people at all. I think that it is used for the purposes I stated, which is to "refer" to blind-testing, as a diversion from the real issue.

I am making a distinction here, between actual blind testing, and the way it is "referred to" in the context of this discussion. It is referred to as "scientific" reasoning that blind testing will show that audiophiles cannot hear what they claim to hear, and that is what I take issue with. It has been my experience that blind testing actually bears out the statements that audiophiles make about hearing differences in equipment. Not in all cases, because some equipment is not different enough for many, or any, to hear. In most cases that I have seen, it is different enough to hear.

I have never claimed that there are differences in ALL cases, and this is borne out in regular informal listening tests as well as it may be in blind testing.

So, I feel that blind testing is not yielding any more information than we already can obtain by simple listening testing. But I also feel that blind testing is being used for an entirely different purpose, in the context of these discussions. In these discussions, it is often claimed that no difference in sound exists between low and high priced cables, or amplifiers, which are the 2 items most often cited in this context. It is then generally stated that if we were to be placed in a blind testing situation, that we would not be able to tell the difference, and thus are wasting money by buying higher priced "audiophile" cables and amplifiers. I feel that since it is obvious, not only by my experience, but by simple empirical inspection, that items made with different designs, and different components will have a different sound, that this line of argument is not related to equipment at all, but must have another purpose. And I have stated what I feel that purpose is.

But it also has another effect. The haranguing of audiophiles that they are imagining things, and "scientific" testing will prove that they are incapable of making sonic decisions, serves only to shake the confidence of a listener, and make him feel that he is imagining things that he is actually hearing. This is a destructive effect, that may in fact lead him to make a bad decision regarding the quality of his audio gear, if he is sufficiently swayed by this line of thinking. If there is truly anyone here who thinks that lamp cord is the best speaker cable obtainable, then they need to take up another hobby.

This is really an extension of a 25 year old battle, which started out in the 70s with the "measurement" people. They read a couple of ads, and thought they knew everything. Then they went around proclaiming that a Technics reciever would be as good as anything you could buy, because it had .0000001% distortion, which we now know was provided by gross amounts of negative feedback, and had hugely destructive effects on the sound quality. But, did this stop the measurement people from running around telling everyone that they were wasting their money on "audiophile" amplifiers? No. In fact they continued to rant about how no high-end amp could be worth the money, when it had higher "measured" distortion than the Technics reciever. And that anything different we heard in these high-end amps was either our imaginations, or it was "euphonic" types of distortion, which is some unknown type of distortion which somehow makes us like the sound more, while still actually being bad. They are still at it with this one. I hear references to "euphonic" distortion all the time on these pages, whereby the implication is made that a tube amp cannot be accurate, because of distortion measurments. After the world caught on to the gag, these measurement people had to "hole-up" for a while. Now they are out again in full force with this again with the cable thing, and they are throwing the amps in again too, in case they can get any more "mileage" out of that one.

While certain measurements may be a decent indicator of whether a component will perform in certain ways, the final arbiter is the ear. This equipment is made to be listened to by ears, and the ears are the judge, regardless of whether the measurements line up or not. The most perfectly measuring amp in the world is not worth a thing, if it sounds like crap. Conversely, an amp that sounds fantastic is worth plenty, regardless of what the measurements are.

This is the same thing as we are seeing here. And the same techniques are being used. Point at a measurement or "scientific" test, and proclaim that anything that you hear that doesn't line up with this data, is imaginary. And to make matters worse, proclaim that if there is no measureable data, that it cannot exist. This is not scientific at all. In the case of blind testing, it is used as a "scare tactic" to make audiophiles think that if they were actually to be placed in a blind testing position, that they would fail miserably, and their entire system is nothing more than an imaginary delusion.

Why would people do this? It serves no purpose but one. To justify their decisions to not make high end purchases, by concocting this story. It truly amazes me that this gets any mileage at all. It is so patently absurd. It flies in the face of what is obvious. They want you to disregard what you experience, and replace it with a reliance on some number, or worse yet, replace it with another school of thought, based upon the "fear" that they try to instill in you that you could not "hold up" under the scrutiny of this blind testing. I find this very objectionable. For someone to imply to me, that I am incapable of making my own decisions based on my own experience, is not going to fly too far with me.

You can all make up your own minds on this matter. And anyone who has the schools of thought that I negatively referred to has every right to maintain them. They have every right to express them. As do I. It is up to you to separate the wheat from the chaff. And the decisions you make will ultimately define what your system sounds like.

Because when you DBT, some people will hear differences where none exist and some will not hear differences when they do exist. This makes any so-called scientific study invalid since it cannot show which is right - those wo say that there are no differences and ones who say there is. It is much harder (maybe impossible) to prove by testing that you cannot discern differences than to say you can.

Why? I think it's because the brain will discern differences in sound when it wants to. It's part of our survival instinct (a police officer chasing an armed person can think a twig snapping is a gun being cocked and will react defensively). Also, not to pick on Twl, if you take Twl and his co-workers informal scientific, but fun, experiment: if I were to tell each of them that I replaced one component when I in fact did not, I would bet that more than one (maybe all) would have identified something as being changed. It's the way the brain works - if it thinks there is a difference, it will try its best to find one (especially with the peer-pressure factor thrown in). Conversely, if it thinks there is no difference, whether out of prejudice or justification, then it won't find one. But if you tell the brain there may be or may not be a difference, then it's up to the listening skills to be invoked - and that too varies wildly among people; to the point where if someone hears a difference in a component might not (and vice versa) in a DBT. That's the controversy IMO.

Gs5556, of course we did that. Do you think that a couple of young audio salesmen would not try that trick? We tried fooling each other any way we could. That was part of the fun.