How to assess and quantify system improvement and changes?


In one of the currently popular threads, a number of members (all of whom I respect) have made significant claims regarding system improvement.

For example: A recent post mentions "a double or triple improvement in my system." A conversation a few days ago with two audio buddies had them quantifying a DAC change as a 10% improvement by one and 30% by another.

This, by virtue of our pursuit, occurs throughout our discussions in the various threads.

Starting this thread to see if we can mine the collective and come up with guidelines and outcomes that are reproducible, relevant and reliable for comparisons and discussions between audiophiles.

Thanks!
david_ten

Showing 3 responses by elizabeth

I think the problem of claims in improvement are usually due in part to the owners closeness and familiarity to the system they own.(though one sometimes hears the same rhetoric about systems they hear belonging to others)
Then the percentages.. Where a claim is of 10%, the actual change from an impartial observer might be 0.1% change. Barely perceptible.It is a problem.. but more akin to fishing stories and the size of the fish that got away.. than any real problem.
One other question is?: Why do you care? It seems the only problem with the inability to quantify the level of 'WOW' is due to wanting information one can use for oneself. Constantly gleaning tidbits in the attempt to be able to use them for oneself? Otherwise who would care?              
To me the main way to do some quantifying is multiple claims in different sources. If several different places have different folks making 'some' sort of improvement claim.. Then I would say OK, seems to be something there.  (Even so, the only way to know is to try for yourself)            
If all I see is one thread (even though it has 2,000 plus posts to it, like some here) touting some product as miraculous.. I pretty much doubt it. (IE if it really WAS that good, plenty of others elsewhere who are NOT connected would also be saying something.            
So what can one do to sort out the claims and the levels of claims being made? Not much. Accept it as part of 'The Human Condition'.
One of the amazing things is hearing something better, when you would never have believed it could be better in that area. Like you are already at 100% and then discover, nope... there IS more to be found. What you thought was perfect got better...