It'll just keep trying as it's not getting the feedback it thinks it should. Yes, there is a relationship between test and spermatogenesis but it will not always be balanced or linear.
Correlation between follicle-stimulating hormone, luteinizing hormo... - PubMed - NCBI
Know that most docs do not know hormones. Their title really doesn't matter unless it says A4M Certified.
Sure, probably no need for surgery regarding sperm count as it only takes one swimmer. The issue is your low T.
There is no reason to accept Low T in this day and age. As long as all else is ruled out via blood work I'd be getting a second opinion re your T level.
How do you feel?
How old are you?
Height / Weight / App. Body Fat %?
Take this test:
http://www.landerurology.com/pdf/adam_questionnaire.pdf