I've used Alibaba.com and AliExpress.com successfully and have also gotten ripped off.
1) Check out the company on scammer reporting sites like FraudWatchers.Org. People will post the contact information for those who are ripping people off.
2) Always do your email communication through the Alibaba message system. Don't send the supplier your email address and don't work with any supplier who emails you directly. You wont have any protection from Alibaba if you do.
3) Not many suppliers use paypal or escrow, but those that do are worth it. Paypal offers protection in case you get ripped off. My supplier charges $20 for me send money through paypal, but it's worth it for the peace of mind. Escrow is a sure bet as well.
4) Don't respond to unsolicited offers. If the offer is too good to be true, it probably is. I let my greed blind me to the warning signs that the unsolicited email made and got ripped off big time.
5) Always, always place a small sample order first to confirm the company is legit and the product is real. Don't let the supplier bully you into a large minimum order quantity (MOQ).