Its cool to hang with them even if they are drinking. Hanging with drunk people is funny. Tell them if they want to drink, go for it! But at the same time be adamant that you don't want any. Say your reasons. But bring your own drink (coke, lemon aid etc), so you still are having something. If they are really your friends, then they wont wanna pressure you. They'll want you to be comfortable. My favourite sentence is "Nah, I'm good." This lets the person know that I am comfortable, but I just don't really want to take that shot/hit/whatever. It wont offend them, and it will decrease the chances of the aggressive response of someone wanting you to drink (ie. Increased pressure/ demanding to know why/ calling names etc).
Good luck, and be safe!
Also you Americans are weird. Here we drink a lot younger than 21 haha.