Labels are for sauces not for people.
When we label -- categorize -- individuals we limit them, at least in our minds. What's more, we share these labels that we give, thereby passing along the limitations we use to define others.
I don't believe this involves malicious intent, unless, of course, we are talking about gossip. Rather, I believe it's an ingrained cultural habit, our way of trying to maintain order in the chaos of living.