I think it would be fine if you were to ask that of your family. You are definitely right when you say that Christmas gifts are more for the children anyway. Your family should be understanding since money is rather tight for most people these days.
Last year was the first year that someone brought up the fact that they just couldn't afford to buy so many people gifts, so my entire family agreed on a new tradition: just buying for the kids. And not only that, but to make things even easier, all of the younger kids (ages 2-10 in my family) draw names to buy presents for each other. Kind of like a secret santa thing. So now I only have to buy for the kids that my kids draw their names.
What we did for the adults was a yankee swap. We set a limit of $25 and every adult bought a gift of some sort that did not exceed that limit. We put all of the gifts, wrapped and everything, under the tree. Then all of the adults picked a number out of a hat. #1 got to pick any present they wanted and open it in front of everyone. Then #2 went and picked any present and opened it in front of everyone. After that, they got to either keep their present or swap with #1. Then #3 went and did the same thing and got to swap with either #1 or # 2 or keep the same gift. This went on through 20 or so adults until the last one opened the last gift and got to either keep it or swap with any person.
So that is what we do in my family now. It saves people a lot of money and frustration. Especially those who have their own kids to buy for, like me.