I feel ya bro , it seems like the bigger you get the more you get tried.
BUT , just because someone is smaller than you doesen't meen that your gonna take them. Usualy people are cocky for a reason like they are armed or their is more than one of them , and being armed with knowledge is just as deadly as a gun , if you know how to use it. I'm sure that their are some martial arts bros here that will back this up.
I'm not saying you a pussy but it would suck to get you ass kicked in front of your girl by a skinny little runt !!!