Personally, I think that the appeal is, indeed, fundamentally different. There are different aspects that are being responded to in each, such as the 'hard and functional' and 'soft and curvaceous' aspects.
That said, I find a remarkable amount of beauty in each. Please note: I am talking about objectifying bodies. Not necessarily someone I would want to jump in the sack with.
Suffice to say that there have been body types galore in either gender that are just stunning to me - from toned as all get-out to soft and fleshy like some tiny cherub. It is a combination of things that strikes me as beautiful, not a type.
Genitalia? Part of the package. Ho ho. It is interesting to me that if you stand a man and woman side-by-side one set is quite visible and one is not. My interest in the vagina is, as you can imagine, low at best but I can see the attraction. I can also see the humor. Same goes for the penis. Balls are just too funny, but also beautiful in their own way.
Hard and functional does not necessarily mean not beautiful. In fact, to some people, it might mean more beautiful. Like a beau's forearms. Hard, functional, and fucking stunning.