Jfz - Nice approach you have. We did a mixture of this actually.
During round 1 we learned a number of things. Important among these was the fact that the voters (although they could not see the players) could tell the one being used because they can see which one received the CD we wanted hear. So we decided to mix things up as detailed in my previous response to Tbg. Although we kept evaluating the choral track before the orchestral recording, we mixed up which player started each pairing and THIS WAS DONE FOR EACH TRACK. Thus, for any pairing being evaluated we did not necessarily begin with the same player for the NEXT CD used. After round 1, the voters did not know which player was playing at any given time.
I wanted to address your comment about hearing more the subsequent times you listen to a recording. I agree and some would also say that their focus changes during the second or third tries, when compared to the first time they hear a recording. There was really no way to address this equally for all voters. Three of the five voters know the Rutter piece very well as we have used it in previous evaluations. Only two of the voters know the Bernstein recording. We felt it was important for each voter to have many opportunities to hear each track so they can confidently cast their votes. Here is a little more detail on our listening process for EACH TRACK...
1) With the correct input selector and volume levels set, we loaded both players, one with the test CD, the other with a dummy CD. We then listened to a predetermined point on player A. For the Rutter piece this was at the 2:08 mark and at 2:44 for the Bernstein piece.
2) Rewind and listen again but for a shorter time. Up to 1:13 for Rutter and 1:38 for Bernstein.
3) We asked is any voter needed additional listening and, if so, we would repeat 2) above
4) Unload both players, switch CDs (the location of the one being tested is not revealed/visible to voters)
5) Reload both CDs and select the appropriate line input of the preamp and make the necessary volume adjustments as predetermined by the level matching done during the set up for both players.
6) Repeat 1, 2 and 3 above for player B.
7) We asked if any voters wanted to go back to player A, and we would repeat steps 4, 5, 1, 2, and 3 for everyone. This option was done only for one pairing the DCS vs. the Meitner.
8) Immediately vote with show of hands (no discussions)
9) Repeat the process for the next recording.
One more thing to note, for subsequent pairings and even in between each pairing, we also switched the input selector to which the players were connected on the preamp. Even though we were assured by its manufacturer that Line 1 is identical to Line 2 in every way (materials/parts as well as specs), we wanted to vary this too, just in case ;-)
During round 1 we learned a number of things. Important among these was the fact that the voters (although they could not see the players) could tell the one being used because they can see which one received the CD we wanted hear. So we decided to mix things up as detailed in my previous response to Tbg. Although we kept evaluating the choral track before the orchestral recording, we mixed up which player started each pairing and THIS WAS DONE FOR EACH TRACK. Thus, for any pairing being evaluated we did not necessarily begin with the same player for the NEXT CD used. After round 1, the voters did not know which player was playing at any given time.
I wanted to address your comment about hearing more the subsequent times you listen to a recording. I agree and some would also say that their focus changes during the second or third tries, when compared to the first time they hear a recording. There was really no way to address this equally for all voters. Three of the five voters know the Rutter piece very well as we have used it in previous evaluations. Only two of the voters know the Bernstein recording. We felt it was important for each voter to have many opportunities to hear each track so they can confidently cast their votes. Here is a little more detail on our listening process for EACH TRACK...
1) With the correct input selector and volume levels set, we loaded both players, one with the test CD, the other with a dummy CD. We then listened to a predetermined point on player A. For the Rutter piece this was at the 2:08 mark and at 2:44 for the Bernstein piece.
2) Rewind and listen again but for a shorter time. Up to 1:13 for Rutter and 1:38 for Bernstein.
3) We asked is any voter needed additional listening and, if so, we would repeat 2) above
4) Unload both players, switch CDs (the location of the one being tested is not revealed/visible to voters)
5) Reload both CDs and select the appropriate line input of the preamp and make the necessary volume adjustments as predetermined by the level matching done during the set up for both players.
6) Repeat 1, 2 and 3 above for player B.
7) We asked if any voters wanted to go back to player A, and we would repeat steps 4, 5, 1, 2, and 3 for everyone. This option was done only for one pairing the DCS vs. the Meitner.
8) Immediately vote with show of hands (no discussions)
9) Repeat the process for the next recording.
One more thing to note, for subsequent pairings and even in between each pairing, we also switched the input selector to which the players were connected on the preamp. Even though we were assured by its manufacturer that Line 1 is identical to Line 2 in every way (materials/parts as well as specs), we wanted to vary this too, just in case ;-)