Hi Dan. I was at an A'GoN member's place last weekend doing a cable comparison, and while we didn't do the tests blind we did hear the same differences every time. However, I have a feeling if the comparison was conducted blindfolded it would have been much tougher to decern any differences. Knowing which cables were in at the time probably exerted certain influences that we have no way of measuring.
But back to your question, would I do it? Maybe out of curiosity. But I'd be very afraid of finding out that I do indeed have tin ears, and all this spending and research was a waste of time. |
Hi, Dan. I'm glad that someone else is thinking about this. First, I have done blind listening tests on many occasions, particularly at the audio store I used to work at. When business was slow, we would do that for fun, just to see if we could tell what equipment was being played. But the way we did it, we remained blindfolded while an entire system was set up and played for us. We then would try to indentify every piece in the system without ever looking at it. Many times we were successful with every component. When only one component was changed in a known system, it was no contest, and we quickly tired of that easy game.Ok, so we were very familiar with the sounds or the stuff we sold. Fine, but we could tell the difference blindfolded. So, yes I have done it, and yes I could do it.
I don't believe however, that blind testing is the only or even the best way to determine listening differences. When you make a change, or especially a tweak, you don't know which way it is going to go. You cannot possibly assign a mental value to the change. So this idea of the vision of equipment making imaginary sonic changes is at best some form of unvalidated conjecture. You just listen and you tell if there is a difference or, not. Simple. It is empirical.
And, I am not opposed to the simple act of blind testing, if that is what you want to do. But I am opposed to the incorrect supposition that it will lead to some kind of "fact" that there is really no difference in the sound of equipment. And that all this is just "in our heads". That is where I am coming from. |
I have tried it. I have blind tested myself with CD players, amps, cables, etc. Sometimes I could hear a difference sometimes I couldn't. Like you said when hearing it at a dealer, things sound different. It could be the room, the gear, etc. So how can someone tell what they are hearing? When I play with gear in my own system the differences are often small, at least to me. There is another thread that is going on titled "electronics don't matter" that brings up similar issues. So I'll refer everyone who is interested to check that out.
I have to say however I take absolutes like "the best" or claims that differences are "Huge" or "Blow you away" with a grain of salt.
Paul |
SACD equipment is made the way that SACD sounds better but in reality good CD-player can do much better than SACD player so this test for golden ears as absolutely irrelevant.
If you ask me if I ever put my golden ears to the test I would say no since I have no golden ears. I only have my own made up from drum acceptors with a-little hair grown and flexible tissue shell covered by tan skin. |
Setting up a truly objective blind test (one that eliminates all of the biases that scientists claim are inherent in sighted comparisons) is not a trivial matter. Plus, you need at least a smattering of statistics training to properly interpret the results. That makes true blind testing a nonstarter for most audiophiles. And this being just a hobby, there's no need for that kind of rigor.
One important thing to remember is that blind tests can prove there's a difference, but they cannot prove that there isn't, or that you can't hear one. That's a basic law of statistics. (On the other hand, if you do a whole lot of blind tests that come up negative, the evidence does start to point in a certain direction...)
That said, if you're interested in the limits of what humans can and cannot hear, and why blind testing is essential in determining this, you should pick up a basic text in psychoacoustics. Used, older editions can be had cheaply, and it might save you some money on audio gear. |
how exactly would you conduct a blind test? in my listening room? i have no problem detecting differences in any equipment i use, good or bad i'm not predisposed, but in an unfamilar environment there would be just too much new information to process, at least for me, and that is how i see most blind tests conducted and i consider those worthless. |
Good point Newbee. There has to be a "reference". You must be listening to equipment that you're familiar with in an environment with which you're also familiar. I would tend to agree that "double-blind" testing performed by people who don't even know what they're listening for (Twl pointed this out elsewhere) is flawed at best and completely worthless.
What I'm referring to is conducting your own "in-home" blind testing where there is a point of reference and, hopefully, some relatively skilled ears listening for the most subtle of contrasts.
I've listened pretty extensively to my friend's system (almost as much as my own!) so I'm comfortable with using it as a reference. Additionally, my friend doesn't make nearly as many changes to his system as I do, so it's been pretty constant for a long time.
P_mmk also raises a good point about people make hyperbolic claims such as "the best" or "blows you away". I have heard people say such things about SACD which was one of the reasons that I wanted to conduct our informal test. While there was certainly a difference, it was not as obvious as I would have hoped.
Imagine the reaction people had to color television when it came out. Cable vs rabbit ears. Digital cable/satellite vs analog. DVD vs VHS. HDTV vs conventional broadcast. In the visual medium the changes are more obvious.
Ours is a hobby of incrementalism. This is all the more reason that we should be training our ears. It's easy to say that I hear a difference, but harder to prove. I'm not suggesting that I need to prove it to somebody else. I want to prove it to myself. |
Every time I make a change. Sometime during my mis-spent youth I suffered brain damage that affected my short term memory (you know like forgetting where you put your car keys). So every time I switch one cable for another or one component for another, by the time I get back to my couch I can't remember what it is I'm listening to.
Actually, I don't know anyone in my real life who would have any interest in doing the switching for me, nor am I sure I would be interested. I am perfectly happy deciding for myself that the new amp I bought sounds better than the old one. I trust myself to do this, because, for example, I have brought amps home to audition and determined on a sighted basis that if there was any difference, I couldnt tell or it was so subtle that I did not need to make the purchase. I have no interest in impressing anyone or any part of my ego invested in my hi fi equipment (I didn't design the stuff).
What I have said recently, and have heard privately from some real pros, some well known, is that properly designed inexpensive late model amps and cd players get the job done as well as expensive stuff, and no one has published a report of a rigorous blind test that shows otherwise. I wasnt really happy when one of my favorite reviewers told me that there isnt much of a difference between my expensive cd player (I paid a little more than 1/2 the retail) and a NAD. I dont even agree with him. But I wouldnt want to put any money on my belief.
The other thread to which you refer is hilarious. The author turns everything on its head. Scientists and professional engineers use double blind testing all the time, but he calls DBX "pseudo-science." The camps are usually divided into scientific objectivists (those who argue for DBX tests and say all cables and properly functioning amps sound the same) and subjective relativists (audiophiles who believe that everything sounds different), but the author of that thread calls people championing an objective scientific approach "subjective relativists." There is no way to begin to have an intelligent conversation on the subject
I am not one of them, but I respect the scientists and engineers I know who advocate DBX testing of audio components. I dont see how you could find fault with them. They're trying to save you money and redirect your attention to things that really matter - speakers and room interaction. |
paulwp, i'm highly suspicious of anyone trying to save me money when i don't hire them to do so - i really think their pushing their own agenda. while i respect engineers and scientists, as well as physicists, doctors, whatever, remember that science itself is a moving target, that it is constantly changing depending on the developement of new facts, laws of nature, theory, etc. a closed minded professional is much more dangerous that the most sujective idiot as people will take the professional seriously - what he sez can seem to make so much sense(!)especially if thats what you are predisposed to hear.
danheather & p_mmk, couldn't agree with you more re hyperbolic comments regarding equipment differences - when i see this i quickly turn the page - it just ell me that the write has something to sell (philosophically or actually). |
I have been able to tell differences in equipment before. But, like most said, sometimes I can't. Some differences grab your attention and some just pass you by, I suppose.
Off topic but interesting: My friends and I were drinking beer a few weeks ago (not my favorite drink, but I am familiar with it) and we noticed that we had several different brands of which we thought we knew their characteristics. Some were imports, some were domestics, and some were microbrews; both cheap and expensive. We began pouring each other glasses and not revealing the brand. On many occasions we had no idea what we were drinking. And, oddly, some of us said that we liked the test beer and found out that it was a brand we previously stated we disliked.
I don't know...Maybe all that beer made it seem more interesting at the time.
Brad Day
Atlanta, GA |
Doing a double blind test with statistics is quite difficult with audio gear. The double protion of the title is that the presenter also doesn't know which stimulus is being given to the subject (non-verbal clues are not allowed). However, it's much easier with water. After buying bottled water for years, my wife suggested that I subject myself to a double blind test with three different types of bottled water and a fourth choice, tap water. I was confident that I could tell the difference between each of the four choices and that I would have no dificulty identifying my favorite. The results were statistically significant (overwhelmingly so) with my clear choice in the experiment being the tap water (not my 'favorite'). -- Now, having said that, I don't mean to suggest that I or anyone else would have similar results for high end audio. I personally believe that the system I have today, that I very much enjoy, is a direct result of successive improvements made by subjective comparison of alternatives. I only mention it to indicate that the results one gets from such tests can be surprizing and that you too can give it a try with other subjective choices that are easier to conduct as a double blind test. |
How could anyone clinging to an argument which is easily overturned by simple auditioning, be considered "scientific objectivist"? I would offer "dogmatic pragmatist" as a more accurate description. The term subjective relativist does apply to them because the testing used is subject to their subjective "statistical analysis" of their conclusions, which are, in fact, skewed by the attempt to make the analysis conform to their preconcieved notions.
I offer as a refutation, the vast bulk of the audiophile community, who have voted on this matter with their dollars to purchase equipment which does sound better than their previous gear. I cannot accept the (scientific objectivist) notion, that we are all in a state of mass-hypnosis, causing us to make expensive purchases for no gain. This is absurd to say the least. Should we not believe what we see because of "psycho-visual" implications, or disbelieve what we touch because of "psycho-tactile" ones? Why not? We are being asked to disbelieve our ears because of "psycho-acoustic" reasons. But some people are just more "informed" than you and me, don't you know? |
brad and danner, love the beer and water analogies. what is clear to me from your posts was that you could tell the differences, just that your preference were not what you expected. just shows that one could be pre-disposed to a certain taste, and for this subject a certain sound. the objective advocates would have you believe all water, beer and my favorite, wine, tastes the same if you can't see a label. by the way, all white wine tastes the pretty much the same to me because i don't like white wine that much and rarely drink it often enuf to recall the taste of the last one consumed. |
This question strikes fear in the hearts of audiophiles, perhaps because it takes some of the fun away or suggests that we have all spent way too much money on audio gear.
I have been interested in this hobby for many many years and consider myself very keen and reasonably experienced.
So I was horrifed several years ago when I found it very very difficult to hear the addition of a Linn Numeric DAC to a Linn Karik in a friends CD setup, even when played through Mark Levinson amps and Watt/Puppies -- and this wasn't even blind. Perhaps this is a bad example, but this was an expensive, Stereophile Class A/B product that supposedly changed the world of CD playback.
Still, I think I can hear a lot and I know what I like. Having agonized over the double blind question, my conclusion is that there is nothing to be ashamed of in placebo effect.
If we perceive that tweaky voodoo mods and/or esoteric gear makes the music sound better, than it does! And if the pleasure that we derive from that experience exceeds the expense and effort that we invest in creating it, than it's a good and worthwhile thing. I suppose that's why we are all here.
But I would guess that far more golden eared audiophiles would fail double blind tests than they would care to admit. |
Brad,
As someone in the wine trade I have to say that blind tasting is one of the hardest and most humbling things to do. I think beer would be a bit easier than wine but still hard. As far as prefering a beer you previously disliked, I'd say you had one beer too many.
On the subject of blind listening, who has the time! I barely get enought time just listening too my system, much less scientifically evaluating components or, God help me, cables. As long as I enjoy the way my system sounds, I'm happy. At least until I see another shiny toy I must have! |
I don't know what would be more embarassing...learning that (through blind tests) you prefer Bose 901s over your _____(your favorite boutique brand here)...or that you like Miller Genuine Draft over _____(your favorite micro-brew here).
As Newbee pointed out...I think that we all agree that (in most cases) there ARE differences...just perhaps not the same preferences that we might have supposed. |
Newbie, the objective advocates would like you to understand that if all beers, water and wine tested the same (which they don't), they would taste the same. Of course one can tell the difference (to a point) when products are different i.e. more acid, less sodium and these differences can be detected.
ABX tests aren't to determine your good taste. People claim to hear differences and I believe them. What I don't believe is that there is a difference just because they hear one. Example, the same wire is compared to itself and one hears a difference. Of course that wasn't fair, I tricked him/her! Of course there are different sounding wires (and amplifiers), but these are designed in to please a certain segment (customers) and I am not sure that I want products that are not flat in frequency response or that possess other distortion products.
There is are limits to how much a human ear/brain can resolve differences in perceived sound and ABX tests tell one what those limits are. This is essentially the fear of those criticising these objective tests - that it will be revealed that their hearing isn't any better than anybody elses when it comes to detecting differences, although it (the hearing) might be better educated enabling the person to describe the difference, if any exists.
Salut |
I think that the comparison to wine is appropriate. Just like wine, one's ability to discern differences in audio increases with ones experience. I can't tell the difference between a $10 bottle of wine and the best wine because I don't drink much. When I was in high school, my stereo sounded great compared to my clock radio. Over time, however, as I developed more experience with audio, I found that I needed more. I can now easily tell the difference between equipment and will pay for what I think is better. I won't, however, pay $200 for a bottle of wine because I can't tell the difference. |
salut, i'm an easy sale for the proposition that people hear the same sound differenty. people can be predisposed to hear, or ignore, certain sounds. physically, the hearing apparatus of each individual is different and the brains interpertations of the electrical impulses from the ear will certainly differ from person to person. however, where i begin to be sales resistant is when i am told that there is no scientific basis for a difference which people claim to hear, ergo there can be no difference in fact. science is nothing more than a body of extant knowledge and that body grows larger each day. what science can not explain to day they may be able to explain tomorrow. objectivists only see what it to be reaidy seen and do not consider what may be discovered tomorrow which will challenge their opinions. look no further than the various theories and facts regarding evolution - its obvious that the more we know the more we realize how little we knew before, and how little we know about what we will discover in the future. i'm still on the fence regarding evolution as well as audio. all good food for though. |
Pick up any Introductory Psychology book. Turn to the chapter on sensation and perception (usually ch. 3 or 4). You will find dozens, if not hundreds, of example of how expectations influence perception, whether visually or auditorially, ranging from the classic of hearing 'Satan rules' or 'Kill your parents' when playing AC/DC Lp's backwards, to the 25 year old study where the sound of the letter 'g' was removed from the word 'legislature' and replaced with the sound of a cough, and nobody could hear the difference. This is one reason why many people WANT to have objective tests of audio equipment; if we expect to hear a difference we will. People who 'hear' the cough as a G aren't lying; it's the way perception works. You can't help it.
There are a number of good comments above about the difficulties inherent in objective double-blind testing. One that I didn't see mentioned (maybe I missed it) was that you shouldn't do it with only one listener. You need a large group of listeners, with each person randomly assigned to one of (at least) two experimental conditions--one group hears component A before component B, the other group hears B before A. Of course, before you can even be in the experiment, you need a hearing test, ESPECIALLY if you are a male above age 40 or so, like (I believe) many of us here are. Not much point in comparing the treble response of two speakers if you can't hear anything above 10K (or whatever)!. You also need standardized source CD's or LP's, volume checks with decibel meters to make sure both components are equaly loud, signal detection theory when analyzing your data to correct for guessing......
Ok, I'm being (somewhat) sarcastic here. My point is that while we could do this, it's TOO MUCH HASSLE AND NO DAMNED FUN AT ALL!! Where's the music in all of this, man? |
It has been said by others, but I suspect that everyone's threshold for hearing differences is, well, different. When I upgraded from an old Sony receiver to a Musical Fidelity integrated amp, I heard a massive difference. When I added an Arcam CD player versus an old Magnavox 650, I didn't hear much difference but really didn't listen sided by side. I have since tacked on the new MF DAC (thinking that was why I didn't hear much) but don't really hear much difference between the Arcam direct or through the DAC. While not even single blind, I can flip back and forth and I hear something, or maybe I just think I do (the salesman certainly said he did when I took my Arcam to the store to listen ;^).
My conclusion is that there clearly are some differences I can hear but some that I can't but others probably can. Whether it is their training or hearing damage from sitting in the second row in front of the right speaker bank at a Chicago concert in 1974 that left my ears ringing for days, who knows.
At the end of the day, though, I enjoy the music when I have the time to listen. |
For me, listening hard to tell differences creates its own problems. My tinnitus starts raging, and I find that the harder I listen, the less my ears can discern. Over a long term, I can (I think) tell the difference between component A and component B. And, of course, sometimes the differences are dramatic enough to be immediately discernable. But the better the system gets, the more subtle are the improvements. And really appreciating them takes time, for me at least. |
Neebee, I don't need a scientific basis to tell me that there is no difference in sound with the same wire, yet people thinking that a switch had occurred continued to hear a difference! If only a switchless change test could be devised (3x blind?)!
It is interesting that people who claim that science can't explain everything, turn to "science" to explain the differences in sound in cables and other devices.
Salut |
I agree with Newbee about having a familiar reference. If you do a test at a friend's house or a dealer, you are only hearing how a component reacts in that environment and with those components. Since you are not familiar with that environment, what you hear could mean anything. For example: if the unfamiliar room has a very lively boomy quality, you may select a lean cable or component that won't work in a more typical environment, etc. I am always looking for a lower cost item that performs like the big buck items, so I am not prejudiced towards the expensive item. It goes right back to there is no right or wrong conclusion. Every system is different. Not to mention how good or different everyone's given hearing ability is. My ultimate reference is the concert hall. |
As far as wine, I still chuckle about the first time the California wines won a big international competition (blind tasting). All the French judges tried to go back and change their votes! (Hmmm....just like figure skating!!) |
inpepinnovations, i agree that people do tend to look to science for answers to questions they cannot understand, (or a god for that matter). i also agree that some people will hear a difference where no difference exists, i.e. your same wire example. the same can be said for a lot of folks who may think of themselves as objectivists, that is "all cables, amps, whatever, sound the same", and all they have as their basis for this is their experience. they simply have not been able to hear a difference therefor their ego demands them to believe that no difference can exist. since the majority of all audio components are made differently why would anyone want to argue that they must sound the same? yet they do, all the time. then they want to do a double blind test to prove their theory but they end up basing the results on things different than the absolute information provided, that is for example my state of mind at the test, my ability to focus, my familarity with the components themselves, ad infinitum. audio reproduction and hearing are both extremely complex issues that i think are not likely to be resolved by any pratical blind test that i have heard of yet. just my 2 cents worth, but for me it really doesn't matter much what other people can't hear, it only matters what we can hear - and in that context why would anyone with any social grace want to destroy someone else's fantasies about the existence of differences in the sound of components, or god for that matter. |
I guess all I can say is that I have been the test subject in a blind test for cables. I was sure I could hear a difference until I couldn't identify my expensive stuff from my inexpensive stuff. The three other audiophiles I was with couldn't tell the difference either. For me, the claims of psycho-accoustics are real. I "heard" a difference because I had to justify it to myself why I spent that much on a cable. Now, I can't claim to have heard every cable, but I do know that I won't spend tons of money on cables ever again, speakers and room placement is where it is at if you want to tweak your system. |
I thought anything that cannot be understood/explained is God's will. Are audiophiles mostly creationists or darwinists? |
All this discussion reminds me of a situation that occurred at a conference in Chicago.
There was a group of chess enthusiasts who were checked into my hotel and were all standing around in the lobby discussing their recent tournament victories.
After about an hour, the manager came out of the office and asked them to disperse.
"But why?" one asked, as they moved off.
"Because," he said, "I can't stand chess nuts boasting in an open foyer."
Nearly ruined my golden ears. |
Albert, that was bad, but I loved it! Let's move back to beer tasting for a moment.
I had a number of parties where blind beer tasting was the theme and involved at least five couples. We even kept score and rewarded the individual who was closest to their original guess in rating beers (least squared difference & forced spread of ratings). More often than not, people were surprised. They could absolutely tell the difference among beers but it did not correlate well with their day to day preferences.
Why? I think it's because we had to keep the portions reasonably small so people could drive home. Like a too forward top end, it sounds pretty nice in the short term. With the beer, we would have been better off with a couple of bottles of each brew. With audio equipment, you are better off with several hours of experimentation (unless, like Twl, you are already familiar with the equipment).
The problem? With beer, you get drunk too quickly to compare more than a few. With audio, you start forgetting all the nuances (well, I do anyway) of earlier stuff unless it is your reference equipment -- this is a hobby of nuances. Alas, I'm only good at blind tests when taken over a long period of time and with limited comparisions (which is not to say I don't know what I like and I am quite willing to reject some stuff pretty quickly -- the issue is with the nuances once you get past the reject pile). That's my story, and I'm sticking to it! Good post. |
sugarbrie, if i admit to being a darwinist does that mean i can't pray to a god? do i get drummed out of the audiophile corp? i think its safer to try to be a darwinian revisionist while attending regular services. lord, its tuff being agnostic, please help me. :-} |
FWIW- I read people hear more false positives (they think they hear a difference but there is none - ie. wire not changed) than true positives (they really do hear a difference). |
