If everything else is in order (training, diet, etc) having both of them seldomly and in reasonable amounts wont be too harmful, particularly if you're bulking up. Life is short, keep mindful of your goals, but there's no need to become overally anal about every aspect of your life---it'll just make you an unbearable asshole. Try to reserve them for either, earlier in the day, post workout, or on a "cheat day", and don't make a habit of turning to them for comfort.