One of the most difficult aspects of sanity's war of defense against the blossoming leftist cultural totalitarianism is the fact that rhetoric and semantics have been put to much better use by the left. They've long been excellent in this area, of course. The cultural marxists, since their genesis at the dawn of the anti-fascist movement in Italy and beyond, have recognized the fundamental importance of manipulating and adopting language as a tool for social change.
Libertarians are often entrapped by this phenomenon as well, referring to vague notions of liberty and rights without realizing that pretty much everyone accepts these abstractions, each attributing to them a completely different, and often mutually exclusive, definition. For instance, the far left refers to the need to set men free, to give him liberty, from the authoritarianism of social expectations and responsibilities. And, they ask, how can we call a person "free" if he is "forced" to pay for his healthcare?
Thus, everyone believes in liberty and rights, but nobody disagrees with its meaning. From this, it is magnificently simple to control people, to get elected, by merely referring to the catch phrases that spurn our sheepish population to action.
But the problem is not siloed to political issues. Concepts such as neo-nazi, neo-fascist, neo-confederate are applied to almost all ways of looking at the world outside of an historically far-left interpretation of things. And consider the word "racist." Even those libertarians and conservatives who consider themselves brave (perhaps they very much are, in our environment) for standing up for "even the racist's" right to speak, already give credence to the presumption that certain positions are, in fact, racist by definition. If you are hesitant about mass immigration, for any reason whatsoever, you are, by definition, a racist. If you think the entire LGBT movement as basically silly, staged, and propped up by a media drooling for manufactured controversies, it's not that you are wrong or misunderstand, it's that you are "hateful" and "bigoted." Since tolerance and inclusiveness (two other words redefined for power-purposes) are sacraments in our secular theocracy to be labelled as hateful is a terrible consequence of not towing the cultural line.
Let's focus on the immigration for a moment. It's a good example of the rhetorical battle at play. Many conservative protectionist types (and Bernie Sanders) have stated that a mass influx of immigrants is bad for the national economy because it brings down the price of labor and forces current residents to either accept lower wages or to simply take unemployment. This is an economic fallacy, of course. But if this is your reasoning for stopping mass immigration, you are economically misled, not a racist. But since racist is to be applied without rhyme and certainly without reason, so many people simply use this as a drag net to dismiss all those positions that are not acceptable.
Thus, while it is important for us to defend the right of racists to speak, we also need to stop pretending like everything is racist. Of course, this word is less shocking, as it has been devalued, and thus, ridiculous phrases like white supremacist has taken its place. Such that even I, a random black guy on the internet, is in fact a white supremacist. The absurdity speaks for itself.