I am so sick of that retort. More than 50% of the population of this country is made up of households making under $50k a year. Frankly, for a family of three or four, that's dirt.
I worked for an actuary. I knew how much he made, for a fact, and he cleared well over half a million a year, yet he managed to get income tax refunds, when I, a single working mother, was paying out at the end of the year.
The richer you are, the less money you tell the government you make and because the tax laws are so convoluted, you can afford the CPAs and lawyers to help you get away with it.
My simple argument: If rich people and big corporations wanted a flat tax they'd have lobbiests out there pounding down congress' doors. Except congressmen don't want a flat tax, either. People who are in the big money brackets don't want a flat tax because suddenly they'd be held to the same standards everyone else is, the rest of us would probably be paying the same amount.