Yes, how rude of him to inconvenience you by getting sick. I’m sure he does it just to annoy you.
There are tons of reasons some people get sick more easily than others. An underlying health condition (such as asthma or COPD), an impaired immune system, effects of certain medications, nutritional deficits, poor hygiene, having young children constantly bringing germs home from daycare or school, or just bad luck. Allergies can also cause symptoms similar to colds.
Regardless, I don’t think you can blame your coworker for getting sick. And if you usually catch his colds, doesn’t that mean that YOU are constantly getting sick as well? What’s wrong with YOU? (See how silly that sounds?) Make sure you’re taking care of your immune system by getting enough sleep, eating well, and drinking plenty water. Wash your hands often and avoid touching your face.