Why are low impedance speakers harder to drive than high impedance speakers

To me it seems like if you need twice the amount of electrons to flow to a 4 ohm speaker than to an 8 ohm speaker, the amplifiers would need to work harder and in return wouldn't this cause more distortion?  Also, if it takes twice the amount of current to drive a 4 ohm vs an 8 ohm speaker, wouldn't this mean the 4 ohm speaker is less efficient.  However, you would think the 8 ohm speaker would be less efficient because it has twice the amount of resistance to the current.

I was told once a 4 ohm speaker requires twice the amount of current than an 8 ohm speaker.  Because a 4 ohm amplifier delivers twice the amount of current, would this in turn supply twice the amount of information to the speaker to create more detail in the music?

Higher impedance speakers are easier to drive to a point. A speaker with a 1000 ohm impedance isn't going to be very loud. We tend to call those things headphones and they're pretty useless from 10 feet away. Beyond a point, 16 or 32 ohm nominal, you're going to want an additional voltage gain stage to achieve the same power output as an 8 ohm speaker. Another gain stage is going to add distortion and coloration just as much as current demand at some point. 

It also must be appreciated that impedance and phase angle are useful tools to tailor a speaker for optimal behavior. You want the woofer to exhibit a big impedance spike at the port tuning frequency and for it to drop off at the bandpass. You want lower impedances where the driver is least effective. Speaker and amp interactions are too complicated to sum up into simplistic terms. More often than not I think the answers to questions like this are "It depends....". 

Since the OP thinks in mechanical terms, this analogy may best describe how speaker loads affect an amp.
Imagine the amp is a somewhat fragile flywheel that will fly apart at a certain RPM. If you apply a resistance (an 8 ohm speaker load in electrical terms) to the flywheel that is sufficient to prevent the flywheel from reaching critical speed it will not break. If you remove part of the resistance (think 4 ohm speaker) the flywheel will speed up. If you remove enough of the load, you eventually reach a point where the flywheel is spinning so fast it fails. Replace the concept of the flywheel speed with power output from the amp. The power an amp will produce is inversely proportional to the resistance to current flow. More resistance keeps the amp in check so to speak.