The whole let estrogen be what it will be is definately a meathead thing said by people who cannot afford Aromasin. I say Aromasin because the top reason people hate AI is they OD with Letro. Second reason to hate AI is "cannot afford it and eat".
You are correct on the prostrate. It also severely impacts the heart longterm. I am willing to bet the same people saying no AI are on the sleep issue thread.