Entirely speaker- and room-dependent, in my experience. Not the same speakers, but my Wilson Benesch Arcs sound the same, grills on or off, so I leave them on for protection. On the other hand, the grills on my BMC Purevox can have a bit of a blanketing effect on the sound so removing them definitely affects the sound. Go with what works, to your ears, in your room.
Anyone Using Harbeths with Grilles off?
Curious whether anyone prefers their Harbeths this way. I would imagine it's room dependent-- they are obviously voiced with grilles on but if your room is unusual (like mine is) maybe they sound better this way. The highs are getting eaten up by furniture and old ears and a strange loft type room at my place so it's working here.