I bought a show-girl dress at a charity shop, I still use it. Charity shops can be a great source of props. I was thinking of doing an "Alice" shoot so I'll probably research on-line fancy dress, I often buy a size too big and use clothes pegs to ensure a snug fit - the camera never lies!
I've hired costumes from local theatres. So long as the costume department are not run off their feet (as they are at the moment with it being Panto season) then I've found them to be very helpful and friendly. It's always best to explain that you want to hire for a shoot, rather than a party. They'll worry less about damage or cleaning bills.
Once you get friendly with them, you might find them willing to sell off odd bits and bobs. For instance, I own 18 Nuns costumes, from a production of The Sound of Music.
I've also bought a few props and costumes off ebay, shot with them, then resold them on ebay. On one occasion I actually made a profit.
PS. I'm also interested in quality props if anyone has any leads.