First off are you eating 20g of good fats or are you eating 20g of saturated fats? Avoid Sat.Fats like the plague, but If they're good fats like olive oil, flax, peanuts, you really don't have to worry over these. These help a great deal.
But in order to tell you how much fat you're eating we need to know how many carbs you're taking in. If you're eating plenty of carbs you can slightly increase fat a little bit. If you're going very low carbs you can increase fat even more so.