Roman Catholic priests are supposed to be celibate. There are several reasons for this (most of which I don't personally agree with, but....). It is true though, that celibate people can often put that sexual energy into other pursuits. I have known celibates who could accomplish amazing things.
Also, sex is not just physical, as you seem to be suggesting. It is also emotional, and at it's best, spiritual. For Catholics, marriage is the union of two people, becoming one, and sharing their bodies is part of that It is not merely for recreation. In that, I tend to agree with the Catholics.
Catholics are taught that sexuality is a gift from God, but, not one to be misused. (I took a course in Sexual Ethics in a Catholic seminary, and that's what I was taught.) But, priests are required to refrain from sex, and the Pope is a priest, so, the rules apply to him to.