We can tell the direction of a sound source by the arrival time difference between our ears.
A sound source 15 degrees to the right of straight ahead at a distance of 4 meters will reach the right ear slightly sooner, but this is not all that clues us into where the sound is coming from. The sound will also be louder in the right ear because there is a more direct path into the right ear. It's more than likely both things that help us determine where sounds are coming from- time differences and volume differences.

