Regular milk has alot of saturated fat, which isn't very healthy if you drink alot. But skim or low fat milk should be fine when taken in large quantities for bulking purposes.
I wouldn't worry about the health risks of milk. You're right Audio, its just propoganda of tree hugging, vegan hippies.