Wrong, HCG was initially developed for women, it's a drug to trigger ovulation, it acts as LH in the body, even though LH does stimulate the testes to some degree that is not what this drug was made for.
https://en.wikipedia.org/wiki/Human_chorionic_gonadotropin
You are still taking exogenous testosterone, even though you may induce some form of natural testosterone production by taking hcg your body isn't demanding it on its own.
Your Dr is a moron imo, if you're on TRT just take Test, don't throw extra crap into the mix, and usually once you go on TRT there is no recovering back to natural.