Any amount of caffeine can cause bone loss.
The loss of bone is most pronounced in spinal vertebrae
So I stopped drinking coffe , tea , yerba mate, and carbinated drinks of any kind (years ago)
I try to do non stimulant fat burners, but may do short cycles of amp02
It's up to you to put the calcium back into your diet either from milk or calcium supplements or both.
Note that coffee and tea may contain other substances which may affect the rate of calcium excretion