If you swim in the AM and lift in the PM you should be fine.
If you think swimming after lifting is hard - even harder is lifting after swimming. I do a masters group (90 mins) 3x a week. I also start my mornings with pullups and pushups.
One time I tried the push/pulls after getting back from swimming. I couldn't crank out nearly as many.