The reason I care is, admittedly, mostly superficial.
How many truly successful people do you see that are fat slobs? Very, very few. As unfair as this might be, it's a fact of life. Society believes that appearance transcends other qualities like 'how beautiful you are on the inside'. So if being attractive gives me an edge, I am going to try to maximize that, end of story.
I also believe that if you take care of your body, people recognize that. They might not understand or condone how seriously you take it, but they respect the fact that you care enough and have the dedication to treat your body like your temple.
That being said, there is a fine line that a lot of people don't recognize and take it to an extreme degree (anorexia nervosa, muscle dismorphia, etcetera). If you can find a balance that you are happy with, and if people give you shit for it, then you have to shrug them off because they are negative influences that you just don't need.