As far as jazz goes, the Sun Ra suggestion is a good one...so is Ornette Coleman, and some of John Coletrane's later stuff...especially "Ascension".
Captain Beefheart and Johnny Dowd are both good suggestions, though you may never get used to Johnny Dowd's voice. For a real freak show, check out The Shaggs. Todd Rundgren's "A Wizard A True Star" is a goodun', but not necessarily a great recording (recorded in 72 with lots of tracks). Neutral Milk Hotel and Giant Sand are also worth checking out. NMH is a bit lo-fi, but the last Giant Sand album ("Chore of Enchantment")is really recorded pretty well. Beck's "Mutations" is interesting, but mellow. The Flaming Lips' "The Soft Bulletin" is pretty impressive. The Beastie Boys' "Paul's Boutique" is a psychedelic rap classic (not a lot of competition there). If you think you'd like an odd Beatle-ish/Rundgren/Neil Young influenced band (with a singer that doesn't sound like any of them), you might want to check out The Loud Family "Plants and Birds and Rocks and Things", "Interbabe Concern", "Days For Days", or "Attractive Nuissance".
That's all I can think of for now. Some of them are good recordings, but the true eccentrics don't often get handed a big budget.