How much fat do you have in your diet? Olive oil, as you said you were getting, has really helped me to get my bf% down.
As far as the carbs go, just make sure what you're eating are low GI and healthy sources of carbohydrates. You're probably getting a bunch of carbs already without even...