2 channel system?


in a 2 channel system (cd player or integrated amps) which component affects the sound the more? im looking to upgrade gradually and im wondering which component will have a more direct impact on the sound? i understand that matching of components is very important and all parts play an important role in the overall sound? but in your opinion which one affects it more? thanks in advance.
inspiral
All very good suggestions. But don't forget about speaker cables and interconnects, either. As someone who first started out with lower end Monster, MIT, and XLO cables and is now using mid-line Transparent and Audioquest cables; the differences can be quite amazing. Your music takes on a new dimension; and details you never heard before are clear and prcise. And as Nrchy mentions, do have some fun and enjoy the great music!
I third or fourth the source recommendation, what's lost(or not read) can not be recovered down stream. Buy the best source you can this way there is no guessing about what the recording is suppose to sound like. For a while(recently) I was running a $9k cd player into a pair of monoblock amps that can be picked up used at agon for under a grand(they are no longer made) and it sounded much better then when I was using my $5k amp with a cheap CD player- and guess what deductive logic will tell us that combining the two(good cd player and good amp) yielded the best results. Avideo brings up a good point about not forgetting cables, once you've tried good cables you simply can not go back, it is often what makes or breaks a system. One piece at a time and it'll be done in no time :)
I'd suggest going with amp/preamp first. I only say this because it is good to see what the source you currently have can sound like. It's hard to judge sources with bad preamplification/amplification. Get amplification/preamplification that matches your speakers.

KF
I agree with amplification first, and if separates, the preamp before the power amp. What good is a great source sent through inferior amplification.

I also agree with the cable comments. Most people put a cheap cable on a lower cost source. You'd be surprised how good a lower cost source will sound connected with better cables.
You said that you wanted to upgrade the integrated amplifier and did not wish to go with preamp/amp.I would suggest you get one of the following: Rowland Concentra I or II,Gryphon Tabu Century,Gryphon 2100 or 2200,YBA Passion Integre.Do not stay in the middle,no source will help you achieve a good sound if you do.Then try different players and judge them accordingly.In the beginning this may sound like a waist of money but in the long term,I think,you will be pleased.CD-players come and go,good amplification stays.That's what I would do.