You can use someone’s person or position to evaluate an argument, but not to dismiss it entirely. Everyone has a right to weigh in on anything and their argument has to be considered on its merits.
Examples
“If you don’t have a uterus, you don’t get an opinion about abortion.”