Yes, I absolutely agree with you, and with monkeymanelvis's answer above. Personally I don't care whether I get thumbs down as long as I get my point across, and have given an informative or helpful answer. I expect thumbs down for some answers which are purely a matter of opinion, but it is rather rude to give thumbs down for a factually correct and / or helpful answer.
There does seem to be one or more users who specifically single me out for thumbs down, and even reporting. I've had violation notices recently for perfectly good answers that were reinstated on appeal the same day.
As I said, the thumbs down don't bother me, but getting violations for questions or answers which clearly do not violate the guidelines is rather annoying. Reporting good and valid answers is not big or clever or funny.