It will help. It will definitely help more if it is a big dog. Small dogs are terrible about marking because they are so hard to catch in the act. Neutering is a good idea though because it will make him less uptight. He can smell a female in heat from over a mile away, and things like that are frustrating to him. He will be much happier and healthier if you neuter him. If you want him to look like he's "got a pair" ask your vet about neuticals - there are a couple of different kinds, and one looks and actually feels totally natural. I have a dog with neuticals - long story, but they look natural. Anyway, getting rid of the smell where he has marked will help too - try cleaning with KidsnPets - you can get it at a Safeway, or order online from Amazon.com - it is a great cleaner because it uses enzymes to dissolve the urine and the smell.
It kind of depends on why your dog is marking, but neutering will definitely make a difference. Remember it will take a little time before his hormones are gone, and depending on him, it may take a while for him to realize they are gone. This is a good thing to talk to your vet about. Good luck!
As a final suggestion, the only thing I know cures marking is a two week period of "crate training" in which you are absolutely consistent. This works. It isn't easy on you or your dog, but it is better than tolerating marking for the rest of his life.