Marathon runners wear diapers, they just let it go. Not suggesting you do this. Seriously, it sounds like your stomach muscle are contracting too much. This could be through anxiety, in which case an easy remedy would be to take the amino acid theanine , or a cup of green tea, before exercise as it contains theanine, responsible for creating alpha waves in the brain (the type that assist concentration and are produced in athletes before a big competition). The theanine content is the reason why you feel relaxed despite the cafeine content of the green tea.
What intensity are you running at? This could have a bearing on your problem.
At higher intensities, the blood is diverted from the parts of the body where it is least needed- for some muscles around the stomach will be starved of blood (which assist the contractions that make you need the toilet), however muscular contractions in that region might increase under such a protocol with sme people and the reverse might be true . Perhaps the solution might be to see whether upping or lowering the intensity might have the desired effect.
Saying that, make sure you schedule your "pit stops" befor the workout.