If you mean DIETARY fats, then there is no requirement for saturated intake. However your body will use mono and poly unsats to make the saturated fats that it needs for good health. Funnily enough, the only dietary fats you need are called "essentail" fatty acids! These can be turned into any other fat in the body.
Diets high in saturated fats lead to an increase in prostate size in male rats, but does not affect the androgen axis. So bigger balls, maybe, but not necessarily more free T floating around. Vegans tend to have slightly higher SHBG and therefore slightly lower free T, but this is almost certainly not due to a lack of sat fats so much as a lack of adequate total fat, total protein and possibly total calories (amongst other major lifestyle differences.....).