Sex and gender are different things.
Sex is biological, it's in your genetic makeup and that will not change. Sex isn't even limited to female and male, humans can also be born intersex (known as hermaphroditism in the animal kingdom) where they may have ambiguous internal or external sex organs.
Gender is more psychological. It has to do with what you feel. Your gender identity originates in your brain, that doesn't mean people choose their gender identity, just like how people don't choose their sexuality. Some people can choose a specific word; a gender, to describe what they feel and who they feel like.
So, whether a trans-man has a penis or not, does not affect his gender at all. His gender is male if he identifies and feels so. But biologically speaking, if he was born a female, his sex is female.