"Making fun" of someone is not racism.
Racism is the belief that you can tell what a person is good for/not good for, or liable to do/unlikely to do, just by looking at them. (Judging a book by its cover.)
If your "fun" consists of suggesting that someone's appearance (skin colour, hair type, body structure) renders them incapable of certain things, or inclines them to behave in a certain way, then you might be expressing some racist beliefs that you've always taken for granted and never really thought about.
But if you're just saying somebody's looks are such that they'll never get a date, that's not racism.