I guess it depends on the person and your social circle.
Considering the times we live in, I wouldn't think a 20-something virgin is odd. I don't think he and I would be a good fit, but I wouldn't think it was odd.
I think if I met a 30-something virigin, I would think that was weird.