1. You may, for various reasons, be seeing what you expect to see. We are not always the best at estimating ratios when we're looking at large numbers of people (for example, studies suggest that many/most people will look at a crowd scene that is about 70% men and read that as being "equal" numbers of men and women, and a crowd that actually is equal numbers as being mostly women)
2. Men can hide extra weight better. Between their greater height, their generally larger overall size, and differences between typical male and typical female clothing (and possibly other factors I'm not thinking of), an average man who's 40 pounds overweight will look skinnier than an average woman who's 40 pounds overweight.
3. Men tend to have more active metabolisms. They build muscle a little more easily, they "burn hotter", and in general they can use up more calories from their food than a woman of similar size. So if a man and a woman have similar diets, the man will have less fat than the woman.
4. Women are inherently/naturally a little fatter. Women have a layer of fat under their skins that men don't, so an equivalently fit man and woman will have different body fat percentages. This likely translates to, well, more fat women than men even given pretty closely equivalent habits.
5. Pregnancy. Women who were pregnant often have trouble losing the "baby weight" afterwards, especially if for whatever reason they can't or don't breast-feed.
6. Social pressure. Women have more general social pressure to "look good", but on the specific matter of physical fitness, and especially muscle mass and/or strength, it's men who have most of the social pressure. Especially where machismo is a significant factor, how strong a guy is is, well, very socially important.
And possibly other factors I'm not thinking of.