Good point here. Quite insightful indeed.
I don't wear button down shirts much, but what I did notice that fit quite nice is the Custom Fit Ralph Lauren Polo. They've got tons of flashy colors and match well. I've had to get them tailored, but at $20 a shirt, it's worth it.