I'd say, consistently, over the years both these lines have had their own "sound", relative to one another - strengths and weaknesses between them both. The Denon has traditionally had a more warm/full-bodied sound, with a bit more bass "umph" and weight, while slightly more relaxed top end. The Yamaha's are more neutral in coloration throughout the line, sometimes a bit cool, if hollow wounding through out the midrange, and a tad less weighter and meatier through the bass reigion. Also, the Yamaha was perhaps a bit more trebble prominent, but I would not say "bright".
I've sold both these excellent product lines in several high end stores, and custom operations, as well as owned products from both companies.
I would usually prefer the Denon's in the lower budget price ranges, for both music and HT (although the Yamaha's did do a great job with their DSP modes for movie enhancement - but, Denon's better 5 channel stereo mode for music was supperior for non-stereo music sound), as I believe, on their own, that they are stronger, more well balanced receivers, that mate better across the board with a larger variety of speaker choices. Also, more specifically within the last couple of years, Denon has also offered the absolutely hands down superior DSP EQ circuit from Audyssey labs in their entry products, which has made those pieces unequaled in value for a budget receiver line!
For the flagship and just bellow receiver pieces through both lines though, I would say the yamaha's and Denon's start to close together more in terms of performance. And certainly, Yamaha has made some great sounding receivers in the $2k-$3k price range sonically, if not equalled or slightly bested by the Denon's. Basically, the sound quality gets a bit closer in refinement on up the price scale...
HOWEVER, in all fairness, in the right application, it has been my experience that the entry level Yamaha's make simply excellent preamp/processors, as stand-alone units, while you drive the pre-out's with outboard amplifiers! The sound here has most favorably been very clear, extended, detailed, sound, matched with much stronger dynamics and bass impact and balance, etc, when configured as such! (on their own, however, most budget receivers are confined to driving small monitors as "small", with limited dynamic power output, otherwise.
I have set up several systems using Yamaha's $500-$1000 AV receivers, using them as strictly processor/preamps in HT/music systems, mated with 5 channel amplifiers, to GREAT EFFECT, over the years! Very nice sound, I would say for what it is.
Still, for what Denon receivers offers on their own, I would lean towards them for anything south of $2k these days, for new! That's just my bias. I would either be looking to run smaller speakers as "small" (100hz to 80hz cuttoff), and likely looking for a deal on a better power amp to drive my system, yes! Anything above those prices, and I'd say you'd not lose with either company. Denon's perhaps a little stronger, with the superior Audyssey EQ.
Both would do stellar with outboard multi-channel amps to suplement, so strongly consider for more realistic high end sound.
Good luck
