It depends how you define "right". People on the lower-income scale have very little discretionary income. All of their earnings go to pay for rent, food, transportation, clothes. Adding an expense burden of taxes means that many of those people will need government assistance (housing, food, etc.), or more than they now get, just to live. So, we'll have them pay taxes only to give it back in aid, plus the cost of collecting the taxes,and administering and monitoring that aid. That's expensive and inefficient. To me, that's not right, and it's better to not have lower-income people pay taxes in the first place.
Also, the way the US income tax structure is laid out, higher-paid people get more kinds of tax breaks than lower-paid people (capital gains, inheritance, other write-offs), and so imposing a tax on lower-paid people is not right in the sense that the burden is unequal and more difficult to bear by the lower-income people.
So, what do you mean by "right"?