No one is debating human rights or equal rights for all groups! What people are debating is whether current sex based protections and definitions should be redefined by gender or not. https://twitter.com/jameelajamil/status/1325071886598823936
Of course everyone is entitled to equal human rights. But should definitions and laws that rely on sex be changed and should everyone affected have a say in those changes, that is the debate.
That includes people for whom gender is a part of who they are, and people for whom it is not and who rely on sex based definitions as a protection against discrimination.
If the definition of woman changes from adult human female to someone with a female gender identity/innate sense of being a woman many of those currently classed as women stop being women. And they have a right to speak about that.