I've been hit hard by people who were 150 lbs as well as those well over 300 lbs. I'm a pretty quick guy and at 290 I can still kick head level and run a 57 second 400M sprint.
The key is to do both. If you are doing Martial Arts hard and Weight Training...the Martial Arts is not going to allow your body to fully grow like it could if you were ONLY doing the weight training. It will BENEFIT you.
Once I started training in the gym heavy...my Martial Arts performance shot through the roof. I could jump like you wouldn't believe.