I wrestled with a similar problem when I owned Avalon speakers. I felt exactly the same despair about the whole audiophile thing--why did I do this? I thought something was wrong with my hearing, had hearing tests, read a lot of stuff on the Internet, etc. For whatever reason, that problem seems to have largely disappeared, though so have the Avalons.
There are certain frequencies that seem to overload my ears with many systems and even with live music. Vibes, for example, and classical soprano vocals. (I think it all dates back to the drum solo during a free Poco concert in Washington D.C. in 1970, but that's another story.) In any event, my advice is to be patient, remove the pre-amp for a while to get relief, then put it back in a few weeks from now and see if your ears still hurt. Experiment gradually, don't panic, and don't be discouraged.