Clomid is composed of two isomers, one of which has more pro-estrogenic characteristics than the other. It's uncommon but could be causing your nipple symptoms as it has a listed common adverse reaction of "breast tenderness". I naturally assumed that this applied more to women than men but it doesn't specify which and I haven't looked into.

Definitely talk to your doctor and ask him (or her) about trying tamoxifen instead. Keep us posted.