When I was really little, I thought skateboarding was cool but never got it down then my board caught on fire. Damn.
In retrospect, I'm glad I never got serious about it. Skateboarding, BMXing, and all that shit just tears up your body. I want to be able to play with grandkids someday.
Also, unless you're someone like Tony Hawk, the drawbacks to skating far outweigh the benefits. You've got the physical beating that skateboarding offers to your joints, tendons, etc. plus the actual cost of skating (boards, wheels, etc.). Unless you're making money off skating, its not worth it.
As for inline skating, as long as youre not doing stunts on a ramp, it should probably be a safe fitness activity.