Best practices when conducting a DAC comparison

David:  I think most of the advice above on level-matching and controlling your own bias is good, of course.  I'll throw this completely subjective, unscientific, and un-critical "test" out there...

Assuming you are not solely comparing at an approximation of "live" or "reference" fixed level, pick some of your favorite music, whether or not "audiophile" stuff, and let it rip.  Which gives you a greater feeling of wanting to turn it up (this, of course, assumes, at that level, the rest of the gear is up to the task and isn't a limiting factor by clipping, compression, etc.)?  Try it several times on different days and alternating which DAC gets played, first.  Just listen without thinking too damned much.  Which feels better?  Once set, go ahead and break out the lab gear, too, to see your peak spls, output voltages, etc.

Yes, it's flawed, but, In my view, this, or some similar "test" you put together, matters.  It matters at least as much as thinking one can tell you more definitively which of the singer's front teeth got smeared with lipstick.

My $.02.  Be critical, but have fun, too!

@georgehifi  and @gdhal  Thanks for your thoughts on the level matching point you are making and are mainly in agreement on. 

@stfoth  I like the 'enjoyment' part of your assessment. Your primary suggestion on how to go about the evaluation is valid and has good company in that it has also been offered up (similarly) as an approach by a couple of others. 

My way to audition a new DAC is with music (a) that I am very familiar with (b) with a mixture of recordings that I love and hate on the existing DAC. Recordings one loves should sound equally as good or better with a new DAC. Just as important is when recordings that grated on you with the old DAC no longer offend and become interesting to you ; these are signs you have fixed a broken area in the old DAC. The pop genre tends to be sprinkled with recordings which excite bad sound on lesser DAC's.

Most DAC improvement is in increased detail   and reduced digital artifacts. DAC's can have tone color signatures (lean/lush/neutral) which will also affect your preference. DAC's also cannot produce what doesn't reach them correctly, anemic bass and blurry detail for accompanying instruments can be signs of jitter issues with the source solution feeding the DAC.

I have MQA in my beloved DAC, and it's disabled. I find I prefer high resolution with a different filter than it demands. I don't hear a benefit of MQA. Sorry.

As for auditioning, make sure both components are 24 hours warm and broken in. YMMV, but that's a more fair test. Ice cold, some DAC's do better than others.

Next, compare red book as much as high resolution. Don't assume that the DAC which has the most difference is the better one. The last generation of DAC's play red book exceedingly well. The previous generations only played high rez well.

Best,

E

I agree with bigkdz.  If you can't tell which one is better within the first 30 seconds there really isn't a significant difference.

Level matching may make a fair comparison if you are switching back and forth, but if at that point you are straining to hear a difference you're wasting time.

Another approach would be to play a familiar cut for a short time at different volume levels, then do the same for the competition.