A week of little work deconditions them, though.
Ask pretty much anyone who squats multiple times per week whether they still get leg soreness from squatting. You'll find that, despite typically doing a greater workload over the week compared with a once-per week squatter, they'll get better growth and strength combined with minimal to zero soreness.
You're correct in that muscles only change in response to a stimulus but they also regress in the absence of the stimulus. Provided you're not someone with piss poor recovery ability, you'll get better results from a workout that hits the muscle groups multiple times per week. Of course, it's always possible for anyone not a newbie to blast themselves into a state that they're unable to work multiple times per week. This is typically not optimal, though.
That said, provided one works for some well-defined progression, once per week workouts can be very effective. For almost everyone, more than once per week is more effective but can be harder to manage.