It depends. Would you like it if your kids did the same to you? If the answer is NO, then it is better to stop doing so now. Otherwise, you will do it with your spouse and your kids will learn the behaviour and when they grow up they will do it TO YOU. It would be better to learn to be tolerant with the people you live with and accept that they can never make you happy 100% of the time even if they tried their very best to do so. When your parents make you unhappy, try to overlook this by thinking of the many times that they have done things that made you happy. So long as the happy things are more, then you should be able to forgive them.
But if you are being silent because you want to avoid making matters worse by talking, then it means that you are being respectful and careful not to say things that might hurt other people's feelings. In this case, it would be acceptable to remain silent when people talk to you.