Cornucopia listed some great websites. I use a lot of thsoe for curriculum and teaching ideas.
Classroom direct is good. cheap shipping!
I ran across one the other day -- which looks pretty good. (and cheap!)
www.teachersparadise.com has some good prices
Also try to think outside the box too. Use things that aren't necessarily 'teacher' related for your classroom.
For bulletin boards -- use bright material. I bought bright purple material with some stars.. it never fades, and you can buy cheap $1/yard material at WalMart. I always try to find material at garage sales/thrift stores. Just an example.
I buy the big pack of "bright" paper and cardstock at Office Depot or you can probably find soem online even cheaper. I laminate a few of each color and mount them across one wall. I attach a "stikki clip" (the wax on one side and little clip thing on the other) at the top. These are colorful on the wall empty, and a place to hold a display paper -- spelling test, drawing, etc.) from each student. I use the paper/cardstock for everything..... Signs on my door, Name plates on the desks, etc. It adds a lot of color to the room, something that simple.
Hobby Lobby has lots of cool stuff -- in every department almost -- that can be used in the classroom.
If you want/need a rug, you can find a local carpet store. Most are willing to donate. Also... look at the students' parents in your classroom. Find out where they work, and maybe you can score some donations like that. Also, in a parent newsletter, you could list some things that you need. (markers, playdoh, etc.) A lot of parents will donate some stuff if they just know what is needed.
If you have a Dollar Tree nearby, they have some teaching materials that are obviously cheap -- and they are cute! I think you can also check: www.dollartreedirect.com