You misunderstand the point of this paradox. By default you become intolerant when you start “removing” people. it is explicitly not a justification for whatever action you claim moral superiority on.

Since almost every political decision will affect at least some fraction of society negatively (even if it would ethically be for the greater good), you can carelessly throw this around to eliminate any opponents for this arbitrary tolerance reason. The only way to make sure the “removal” is fair, as a society absolutely needs these tools to function, is to clearly outline the case when it needs to happen and bring the barrier such that those capable of improvement do not get ostracized into further radicalization. And that barrier needs to be significant.

You bring up “fascist”, at which line does it happen? Genocide execution, support, inaction, Swastika wearing, illegal membership, legal membership of ultra radical parties, support of conservative oligarchs? What is greedy? Robbery, theft, tax evasion, corruption, cheating with the girlfriend of a friend? What is bigotry? You get the idea.