What matters most in speaker design?

Uniform polar response with a slow monotonic decrease in total power above a few kilohertz preferably with less output towards nearby objects.

Getting there is a function of driver diameters, driver bandwidths, spacing, baffle shape/dimensions, and the cross-over.

There appears to be a point at which drivers are "good enough" in terms of distortion and stored energy.

Provided output levels are limited, you sit closer to accomodate the reduced directivity, and get farther away from the side walls my Linkwitz Plutos ($140 driver cost) are surprisingly close to my Orions ($1400).

"There appears to be a point at which drivers are "good enough" in terms of distortion and stored energy."

Drew - are you saying that Beryllium tweeters in new Revel Saloon 2 or diamond tweeters in B&W are mistake (unnecessary expense)?

I originally thought flat frequency response was a primary goal. And it is, IF damage done to achieve this is minimal.

BUT having listened to single driver speakers with no x-over and then adding different tone controls, [DEQ24/96, Parasound Z-pre, 3-band tone control] to flatten the response, I can say any manipulation of the signal makes the sound worse in more important ways. Assuming the driver F-R is fairly flat to begin with. For example, NOT metal with some crazy break up mode or peaky high efficiency Fostex/Lowther.

I find I can mentally compensate for uneven F-R BETTER than the distortion and phase shifts that come with ANY tone control that I have found. A classic case in point is the Eggelston Andra that switched from an uneven mid-driver with no x-over to a flat F-R but with x-over. Surprise was many listeners preferred the older design.

Also, I am adjusting the signal the "right" way, at line level, not AFTER the power amp where passive x-over parts will color the sound even more. And for those who HATE tone controls, you should be aware that designers build tone controls INTO passive x-overs to flatten bass or add the famous 5dB boost at 100hz.

Expressed here better than I can:
Read Johnnyb53 re Flat F-R

So to answer the OP's question in terms of the driver itself, I will respond by repeating what Seas says:

1) Low non-linear and modulation distortion.
2) Excellent transient response.
3) Good coil excursion.
4) Pistonic response.
5) Large windows in the basket to reduce sound reflections, air flow noise, and cavity resonances.

It's been reported that John Dunlavy thought step response was what mattered most in speaker design.

>11-16-11: Cdc
>I originally thought flat frequency response was a primary goal. And it is, IF damage done to achieve this is minimal.

>BUT having listened to single driver speakers with no x-over and then adding different tone controls, [DEQ24/96, Parasound Z-pre, 3-band tone control] to flatten the response,

1) Unless you were doing some form of reflection free measurement (anechoic chamber, speakers buried in the ground and measured in half space, or a gated measurement of some sort most likely using Maximum Length Sequences and software like ARTA) you were adjusting for flat power response which you don't want instead of flat on-axis response with smooth off-axis curves which you do.

2) Unless you were using a calibrated measurement microphone you weren't adjusting for flat response there either.

3) Toole and Olive have found that listeners prefer flat on-axis response with smooth off-axis curves regardless of nationality and preferred musical genres. Olive turned that research into a speaker ranking formula which correlates well with subjective listening tests.