Well, your muscles don't go anywhere, they just lose hypertrophy
The vets on the board are more qualified (and I suspect this is one of those "your mileage may vary" sort of things), but when I go on vacation if I don't keep up with some sort of regular weight workouts I notice my arms start to lose definition and size within 10 days or so. I'm long limbed, so it's tough for me to get any bulk on my arms to begin with, and it's the first place that goes down.