this is, for the most part, a myth, but does have some basis. It is as follows:

when we're still in the embryonic stages of development, we are generic; that is not male or female. Although a male embryo has a Y chromosome and a female does not, this is not visible until the genitals develop. Male and femal geintals develop from the same exact structure (a generic gentalia). For females, there is retraction of the main body to form the clitoris, there is no fusal of the port, leaving the labia around a hole. In males, the main body extends and forms the penis, and there is fusing of the skin to form the scrotum (which will be filled with nuts after birth). Thus, penis and clitoris are analogous structures. The clitoris is the default, but if testosterone is present a penis forms. Thus, penis development requires test. Also, in adults who suffer from various genetic diseases where they are males with high levels of female hormones (high female to male hormone ratio) they have small penises and even penis shrinking upon development of their disease. When you take a cycle of steroids, your natural testosterone production is shut down via negative feedback on the HPTA. Then, when you go off the steroids, your body is not producing its own testosterone, and your female to male hormone levels go way up. Thus, you lose your sex drive and can get shrinking of the penis. It won't shrivel up and fall off, and it can be rehabilitated with proper hormone therapy, but nonetheless these things can happen.

Montgomery