Male colleague suggested I get quicker response from male team members bc I have physical attributes that give me an advantage. Brains, experience and professionalism not on his short list!!!

I thought it a cheap shot.

How would you suggest I should have responded?