It depends on how it effects your estrogen levels. This is why blood work is key. You need to get blood work while on cycle to see where your estrogen levels are. This is what will let you know if you need to be adjusting your AI dose or not.
Personally I like aromasin much better than arimidex anyways. It is a suicide AI, increases IGF-1 levels, and does not negatively effect cholesterol levels like arimidex does.
Remember too that libido can be affected by much more than just hormone levels. There is also stress, etc.