This is a shot in the dark. You said you go to the gym? Are you fit, or overweight? I just did a google of "overweight men and low testosterone" and there were many studies about overweight men, especially with a high concentration of viceral fat, with lower testosterone. Google it and check it out if it applies.
Also if you are overtraining, that could hinder normal hormone production.