This is for Georgehifi especially but others can chime in.

@georgehifi 

Say WHAT!?!?  Where did you get the idea FW amps have no gain? My F5 has about 15.5dB of gain. All but one, the impedance converting F4, have voltage gain. 

Real passives expose the source and the amp input stage to the varying impedance of the volume control. You can try to passively remedy that, but the source and amp are always going to see some wandering load or source. Buffering the volume pot or resistor ladder or whatever you like with something like JFETs provides a very high impedance to the pot and a very low impedance to the amp. And there's no reason you can't buffer both sides of the pot. You'll end up with vastly superior performance across the volume range. 

My F5 has about 15.5dB of gain.

Say yes!!! about half normal power amps.
Which is very low for a poweramp, and why your experience with passives is understandable, and why you need the gain of an active pre.
 
As Nelson said: " "We’ve got lots of gain in our electronics. More gain than some of us need or want. At least 10 db more."  

And there’s no need for active buffers, on passives if there’s no impedance mismatch.

Cheers George

@georgehifi 
As far as I know, the active stage I'm using has no gain. The amp section for it has the same input sensitivity as the F5 which is about line level. If it has any gain it's modest at best. Why are you assuming an active stage has gain? What makes you think 25kOhm is a good input impedance for any source? And what makes you think 15.5dB is half if normal? Plenty of medium power amps in the range of the F5 are 15-18dB. 

Your statement " All passives basically suck" is your own problem with your system/amp gain structure. Live with it.

Cheers George

@georgehifi 
Uh... No... It's a cold, hard fact. Volume pots present a miserable impedance to a source and an amplifier, generally speaking. I could probably feed the F5 with one if I wanted because it's got a 100kOhm input impedance, but if you're dealing with an amp with a more typical 40-50kOhm input, putting a 25kOhm pot in front of it is just plain stupid unless your amp has low enough gain you can run close to wide open. Of course, then you present your source with a feeble impedance then. Sounds very imperfect to me. It's not a solution I'd briefly entertain, much less make my problem.