Impedance or sensitivity?

Depends on the amp. Please tell us if the amp you want will be tube or SS.
Tube amps usually don't like low impedance speakers especially OTL amps. Impedance is hard, if not impossible, for tube amps to deal with when it gets lower. Tube amps can deal with low sensitivity by offering more power. It is costly, but not a limitation in many cases.

So in general impedance is harder for an amp to deal with. Solid state amps don't like high impedance. For example Soundlab speakers will have bass impedance in the 40's causing trouble and possible osillation for SS amps. They put out far fewer watts into high impedance loads - a small fraction of their rating.

It's worth considering too that the impedance spec is nominal. Impedance can dip below it and rise above it at various frquencies, and can have varying combinations of resistance, capacitance, and inductance. So two speakers with the same impedance spec can present an easy and difficult load respectively.

josh358 is right. Just because the stated speaker impedance is listed at 4 ohms does not make it harder to drive then all 8 ohm speakers. The important thing to know is how much and widely the impedance varies. A 4 ohm speaker that never dips below 4 ohms may well be easier to drive then an 8 ohm speaker that dips down to 2 ohms in the highs. Great point!

If the speaker has been reviewed in Stereophile, John Atkinson's measurements at the end of the review will usually provide useful commentary on how easy or difficult the speaker is to drive. That commentary will be based on the plot that is presented of the magnitude and phase angle of the speaker's impedance as a function of frequency, as well as its sensitivity. You can search for a review of a particular speaker at their site or via Google.

The most severe combination of circumstances would be a speaker that has low sensitivity, an impedance magnitude that drops to low values in the bass region (where lots of energy is usually required), and an impedance phase angle that is highly capacitive (i.e., that has a large negative value) at frequencies where the impedance magnitude is low.

Another point to be careful about is whether the speaker's sensitivity is defined with respect to 1 watt, or with respect to 2.83 volts. For an 8 ohm speaker (that is truly 8 ohms) it makes no difference, as 2.83 volts into 8 ohms corresponds to 1 watt. However, 2.83 volts into 4 ohms corresponds to 2 watts, so a 4 ohm speaker that produces 90db in response a 2.83V input produces only 87db in response to 1 watt. 2.83 volts into 6 ohms corresponds to 1.33 watts, which means that a 6 ohm 90db/2.83V speaker produces about 88.75db in response to 1 watt.

At the same time, keep in mind that the power capability of a solid state amp will increase essentially in proportion to a decrease in load impedance, up to some limit. A tube amp, on the other hand, will typically have a much smaller increase in power capability as load impedance decreases, while exhibiting a smaller decrease in power capability as load impedance increases. Those differences between the two kinds of amplifiers can have important implications with respect to tonal balance, as well as with respect to drive difficulty. See Atmasphere's paper on that subject.

Regards,
-- Al