I've never worked anywhere where all the coworkers exchanged gifts with each other. Sometimes there's a grab bag or Secret Santa, but never a large-scale exchange. You could opt out of grab bag or Secret Santa if you want to. I have also never felt pressured to go to an off-hours party that I had to pay for. But, maybe you work in a different kind of place than I do.
As far as what I do, I'm a department head, so I get a gift for everyone who works in my department. They do not all give me a gift back (I don't want them to give me gifts). But, that's my choice: I do it because I want to, not because I feel pressured to. We go to two office parties, both on company time and at company expense: a casual social for the whole company in the afternoon with appetizers and drinks, and a luncheon for the entire division that reports to my vice president.
I've worked for 40 years at 7 different companies, and it's been pretty much the same way at all of them.