Sounds like you have joint laxity and the wraps keep things inline. Without the wraps keeping the joint inline while lifting the slop allows the joint to move out of place and rub where it shouldnt and add additional stress on already weakened tendons and ligaments.
If you chronically keep pushing the joint past where it can handle unaided (ie with wraps) you are going to constantly cause damage and not make any healing progress.
If I were you I would ditch the wraps and let pain be the guide as to how much weight to use. In addition add in 6g (or more) ascorbic acid, 5-6g lysine and 1-3g proline to your list of supplements, spread intake throughout the day. Those are key elements in collagen which is whats needed to repair the damages joints.