are you allowed to say it - definitely, but my question to that would be... 'will he listen' ? Most guys I know would say ^&*$^ that and go anyways, but other men would respect that. If it makes you feel uncomfortable, then you should express that. Who cares what his friends think - they are probably single as well or are in a love-less relationship, or uncommonly in a relationship where they are comfortable with each other and the gf doesn't care that he would go, so he thinks the rest of us should be the same.
I expressed my concern with my bf of 6 years about the same thing. Friends of ours went to Montreal for the soul purpose of strippers (we have none here), and I told him he could go but don't expect me to be ok with it and expressed the feelings after that. He never went :-)
You can't -make him- not go, however a happy wife is a happy life ;)