I can tell after seeing a guy for in the gym for a decent period of time. I don't just assume built guys are on.
For example I just recently noticed a guy that looks like a cage fighter walking around looking alot smaller than he has for the last couple months. Like his whole body shrank. His traps used to pop out abit and his forearms and calves were ok. Now he actually looks kinda skinny to me. Plus he doesn't walk around with as intense of an air about him.
Things like this are what I notice over periods of time. I also tend to notice guys that ALWAYS wear hoodies or sweaters even if you can tell they are built. This is a great way to make it harder to notice fluctuations in size imo.