For records, I second the recommendation of Dusty Groove and Reckless. Dusty is better for soul, R&B, jazz and blues. Reckless will have more rock, indie, punk, metal, and recent stuff. The Milwaukee Ave Reckless is pretty close to Dusty Groove, and it is larger than the downtown Reckless.
Another option for gear is Decibel, on Milwaukee Ave. They carry Harbeth, KEF, Rogue and more. That Wicker Park neighborhood has lots of nice shops, and I prefer it to downtown for wandering aimlessly or shopping. Easy trip up the Blue Line.
May be trickier to get to via CTA, but Logan Hardware is also excellent. And on your receipt they give you a code that allows you to enter the game room in the back. Loads of old video game machines and pinball. Great spot!
