Too much extra activity outside of weightlifting without proper nutrition will hinder gains. Keep it in moderation though, and you'll be fine. It depends on your goals though. If you want to be the next Ronnie Coleman (probably not possible) then I'd say drop the other shit. If you just want to be a bit more muscular, don't worry about it. In fact, boxing will help your upper body.