The caffeine functions as a diuretic, so you have less water volume in the muscles. thus the flat look. Happens to me too. The stress your body feels from the drugs (and/or dieting) increases cortisol secretion. Cortisol has a glucose sparing effect, but its mechanism is to realease amino acids from skeletal muscle, and fatty acids form adipose tissue. If you don't have adaquate protein intake, you can see the affects of being catabolic, rather than anabolic.
Also, the glucose sparing affect functions to keep up glucose levels in the blood, so it doesn't do anything for muscle volume.