You should at least replace what you shut down...
An analogy: Kinda like shutting off the sun outside and then replacing it with dim, green ambient, flourcent lighting... sure you can still see and carry about your business - but the natural sunlight makes you feel better and makes it easier to see and generally more comfortable
A long answer: Testosterone is the endogenous androgen in the body – it flows through your veins every day of your life. It is responsible for many bodily processes. Elevated testosterone levels can improve the overall psychological state, optimize libido and sexual function, as well as create a more anabolic environment for tissue growth. Normal ranges for hormone replacement therapy are around 60-100mg of testosterone enanthate per week. High doses (over 300mg per week) are frequently used by athletes to improve the anabolic effects of the androgen; resulting in further muscular hypertrophy. The Hypothalamic/Pituitary/Testicular Axis will inhibit its own androgen production when it recognizes exogenous support. Anabolic-androgenic steroid (AAS) cycles lacking testosterone can cause problems for the athlete. Using a progestin (for example, nandrolone or trenbolone) can cause a build up of progesterone – which can result in poor disposition and lack of sexual desire in the absence of testosterone. For this reason, even at simple replacement doses, testosterone should be the base of any AAS cycle.