Looks like you are not alone in that "6th member" opinion...
http://theband.hiof.no/band_members/john_simon.html
Al's Wiki info about Al Kooper leaving BS&T not withstanding, the Wiki entry for Child Is Father to the Man puts it more bluntly:
"After a brief promotional tour, Colomby and Katz ousted Kooper from the band, which led to Child is Father to the Man being the only BS&T album on which Kooper ever appeared."
https://en.wikipedia.org/wiki/Child_Is_Father_to_the_Man
All explanations covered handily, no doubt, by the hackneyed "artistic differences".