Labels are important because people can find community using them (which is necessary for survival), and they give others a starting place to understand or learn instead of making assumptions in favor of default identity. There isn't this kind of "why can't we all just be people, I hate labels" backlash against political party labels or familial role labels or religious labels, because people don't feel like acknowledging variety in that way makes them take a new look at themselves. The reason privileged people push back against gender, sexuality, ability, and racial labels (and others related to oppression) is because those labels make one have to acknowledge privilege, which is quite uncomfortable.

If you have to label yourself as straight/cis/non-disabled/white, that is an acknowledgement that you do not suffer the oppression that queer/trans/disabled/people of color do. It recognizes that you have a different experience, and if you are at all aware, that experience is clearly privileged.