New fantastic room but system pissing off neighbor

I recently moved from a typical tight apartment (though with high ceilings), to a large loft space (35x28X14) and the increase in fidelity I am getting is pretty significant. I think it has to do with obviously having to drive the system much harder, actually opening up the volume pots close to or at 12 o'clock, and mostly giving the speakers much more physical room to breath. I seem to be hearing more detail, better individual instruments and into the music much further. This is aside from the expected added stage depth and width I am now getting.

So that's the good news. However, one neighbor (on the far side of the system over 40 feet away) once casually mentioned to me that sound in this building travels. I was surprised since the building is quite old, of stone, and brick, built like a bunker, was later divided into apartments using cinderblock and sheetrock walls, as opposed to just sheetrock like our old place).

So I looked at what might cause sound leakage, and realized the bathrooms share a common vent. I now make sure to close all bathroom doors before listening, and only listen loud for fairly short times in the afternoon, early evenings, etc. No one downstairs, or in the unit right next to the wall where the system is has complained. Not that it matters that much, but since I play only LP's there is lots of down time in between sides, while LP's are cleaned, periphery ring and clamp is installed, etc, so it's not continuous music by any means.

Now these neighbors that were formerly friendly enough when we saw them in the hallways, seem to not want to talk to us at all, and now seem to go out of their way to be unfriendly. The only thing that has changed is that I got my system up and running.

Should I give up on the idea of playing back music at realistic volumes? (we're only talking about 85 db on average or so, I don't listen that loud)

Should I just forget about it and have the attitude that I'm doing my best to keep doors closed, etc, and that noise is part of urban living, etc.

The funny thing is I've never heard sound from any other apartments, except for faintly, when right next to those bathroom vents.

Listening to music is obviously important and therapeutic for me.

Anyone else dealt with similar problems?