Bookshelf speakers and complex/orchestra

My experience with the quest for 'separation' has been good.
I tend to believe the real control of this is in the amplification. If from an LP, then it is also part of the phono preamp. But the amp is the key to clarity.
I have B&W 805s on B&W stands. A Forte 4a amp (owned from new)

For clarity on large orchestral stuff with aplomb try to look for a bigger speaker - something that has large dynamic range and with a critically damped response (a lean tight bass)

The seperation of instruments for example is not as clear as with a slow moving jazz recording. Is this to be expected?

As you add complexity and dynamics it is much more challenging for a small speaker. I would suggest a three way with a mid range of 3 to 5 inches - this will bring back the effortless sound. In some cases, a double midrange driver in Appolito design will help as most midranges struggle at higher levels. Another option is a horn midrange.

Autodexr, orchestral music is some of the most challenging music for any system to reproduce. Systems that sound fine on small ensemble jazz or "singer/songwriter" music can simply fail to resolve orchestral music. There will be many factors affecting this: source, amplifier, cables, as well as speakers. Speaker placement will also be a significant factor.

I'm not familiar with the resolving abilities of your speakers, but don't be too quick to assume the speaker is your limiting factor. I listen to a lot of orchestral music, and for many years I lived very happily with a pair of 2-way Celestion SL700 speakers. Their ability to resolve the inner detail in orchestral music was exceptional. What they lacked was the ability to play very loudly, handle large dynamics or deliver deep bass. But in my 13x17 foot listening room, they were very satisfactory aside from these limitations.

A simple starting point you might consider is to pull your speakers well out from the walls and listen again. Room boundaries can play serious havoc with resolution and instrument separation.

Best of luck!