2. speakers
  3. 1412457273

Why blind listening tests are flawed

This may sound like pure flame war bait - but here it is anyway. Since rebuilding my system from scratch, and auditioning everything from preamps to amps to dacs to interconnects to speaker cable etc, it seems clearer than ever.

I notice that I get easily fooled between bad and great sounding gear during blind auditions. Most would say "That should tell you that the quality of the gear is closer than you thought. Trust it".

But it's the process of blind listening tests that's causing the confusion, not a case of what I prefer to believe or justify to myself. And I think I know why it happens.

Understanding the sound of audio gear is process of accumulated memories. You can listen to say new speakers for weeks and love them until you start hearing something that bothers you until you can't stand them anymore.

Subconsciously you're building a library of impressions that continues to fill in the blanks of the overall sound. When all the holes are filled - you finally have a very clear grasp of the sonic signature. But we know that doesn't happen overnight.

This explains why many times you'll love how something sounds until you don't anymore? Anyone experience that? I have - with all 3 B&W speakers upgrades I've made in my life just to name a few.

Swapping out gear short term for blind listening tests is therefore counter productive for accurately understanding the characteristics of any particular piece or system because it causes discontinuity with impression accumulation and becomes subtractive rather than additive. Confusion becomes the guaranteed outcome instead of clarity. In fact it's a systematic unlearning of the sound characteristics as the impression accumulation is randomized. Wish I could think of a simpler way of saying that..

Ok this is getting even further out there but: Also I believe that when you're listening while looking at equipment there are certain anchors that also accumulate. You may hear a high hat that sounds shimmering and subconsciously that impression is associated with some metallic color or other visual aspect of the equipment you happen to be watching or remember.

By looking at (or even mentally picturing) your equipment over time you have an immediate association with its' sound. Sounds strange, but I've noticed this happening myself - and I have no doubt it speeds up the process of getting a peg on the overall sound character.

Obviously blind tests would void that aspect too resulting in less information rather than more for comparison.

Anyone agree with this, because I don't remember hearing this POV before. But I'm sure many others that have stated this because, of course, it happens to be true. ;
larrybou
schubert
4,867 posts
 
Larrybou, you are exactly right.
Some folks of a scientific bent tend to think everything can measured and/or explained.
The fact our unconscious memories may run the show distresses them.
onhwy61's avatar
onhwy61
4,163 posts
 
From your description it appears that you are listening to your equipment as opposed to listening to music via your equipment. Since no equipment is completely transparent and without fault, if you actively search out the shortcomings, you will hear them. In other words it's your mindset that is creating this situation. The sound of equipment may be a series of memories, but music is something that is completely in the moment. If your equipment is good enough to put you in the moment, then why look for faults? You will only make yourself unhappy.
bhobba's avatar
bhobba
301 posts
 
I am with Doggie on this.

IMHO neither sighted nor blind tests are perfect.

To really get to the bottom of a bit of gear you need both.

But having participated in blind tests I know only too well how hard they are to set up and do properly, which is why you see such a dearth of them.

The thing that always amuses me about blind test aficionados is when I say I am always willing to learn about judging gear can they point me to their write up of blind tests they have done. Can't recall a single one that was able to to that. One cant help but get the feeling for them its religious zealotry.

That said, as I said at the start, you really need both. Its just such a pity blind tests are difficult to do properly.

Thanks
Bill
zkzpb8
208 posts
 
It’s simple - listen with your ears not with your eyes.

I understand the long term exposure to really get to know the sound of something. I’ve heard NS-10s thousands of times - when I hear a pair of those things, it’s a very familiar sound.

But - if you’re trying to make comparisons between two pieces of gear you have to level match, and make instantaneous comparisons - back and forth, back and forth. This is the way our senses work - our hearing is very non-linear, and our senses have a very short memory - much shorter than one would think.
stevecham
3,298 posts
 
We are highly sensitive and quite variable beings of auditory input; our surivival in the wilds depended on it, and especially our sense of hearing, providing information of direction and distance where sight alone often failed. I know not to always trust my hearing because, in all honesty, there are times when, depending on my emotive state, this sense can be influenced by factors that have nothing to do with the system or source material I am listening to. And quantitatively, it seems impossible to replicate or even know when one's emotive state is calibrated relative to an earlier time, when perhaps a qualitative assessment of equipment performance, was being audibly judged. We, I propose, are the greatest source of variation in the subjective enjoyment of music, and this must be taken into account when evaluating one component against another, or a system as a whole, or an artist, performance or recording. We, like the system and the room, play as much a role in the overall test by hearing process, whether sighted or blinded.
More to discover