Well for one thing, you could be cited for failure to curb your pet's waste. It is the law in most cities that it is illegal to leave your dog's waste lying around for others to admire, due to the sanitation and health issues that open fecal material causes (dog droppings spread parsitic worms that your dog can carry without necessarily being directly affected if the dog is treated with a preventative as most pet dogs are. The worm eggs just simply pass through into the waste material. Whats more is humans can be infested with these kinds of parasites as well.).
You also have the fact that allowing your dog into your neighbor's yard without their permission also constitutes criminal trespass, as your animal (property) is considered at law to be an extension of the person. (which is for example, the reason why you can't claim "I didn't kill him, the bullet did" as a criminal defense.) combined with the fact that if your dog ever attacks your neighbor you are positively DONE due to "leash law" which basically says any problem your pet dog causes while off the leash is automatically your fault under strict liability doctrine, because the law expects you to have your animals under control at all times. So in reality, you're lucky your neighbor hasn't called the police, and you certainly have nothing to complain about.