as far as i know, rubber itself is non toxic
that being said, you cannot ignore the allergy issue (some people are allergic and have reactions ranging from mild dermatitis all the way up to anaphylaxis) and other compounds in the rubber product (it is often treated with various chemicals to get various effects eg vulcanisation bla bla which use potentially hazardous agents) as well as the possibility that rubber dust can irritate internal membranes with nasty consequences (inside of lungs = cancer)
so yah. pure rubber in one piece = harmless. chewing truck tires = not so harmless