First Tone controls obviously add Color, there is devices in them that of course changes the frequency response and distortion levels. However rarely do you hear a bad sound or added distortion unless its a loudness button that overdrives the electronics in turn the speakers producing this to audible levels.
Now that being said BALANCE controls are virtually negligible unless a really generic design with bad resistors or something in it, this simply at Zero is making both the left and right signal balanced, but turning it one way or the other is simply reducing gain not adding color or changing frequency response as a whole. The same as adding a resistor to your tweeter in a larger size to tame it down, but fact is it still already has a resistor doing this so is a larger resistor going to add more distortion or just reduce volume? Obviously not add more distortion, so this is WAY overthought.
The balance controls today are fairly transparent so I would not get to excited about "Bypassing" these, but again old generic types could be just cheesy and cause noise etc... Where back then you get the crackle due to simply physical wear and contacts getting dirty.
I doubt anybody with a hi end piece of gear using balance could say they had it modded and it was much better without. Exceptions to the rule I am sure exist, however again its silly, and hey if you have a room that really boosts one channel in a corner to be louder than the other, its much better to have the option to balance it out than not, only other way is many options requiring a new room, or room treatments. I could see putting in a flip switch to pull the balance out just for those concerned, less is more sometimes.
