Yeah, it definitely will have an effect on your body, but I look at it this way: If I'm drinking, there's no way in hell I'll be productive the next day. Meaning, I'll probably sleep-in, miss a few meals, be too lazy to workout, and if I somehow do manage to get my ass in the gym it will probably be a shitty workout. If I do drink, I make sure before that the next day is an off day, or just a cardio day.
And for me, there is no moderation. If I'm going to drink, I'm going to DRINK. Never could have only a few drinks and call it a night.