Conserving hard won liberties and advancing toward a freer society would be easy if everyone wanted to “live and let live” among a wide variety of people, or if the humans who want to suppress, disparage, or punish difference could be educated or acculturated into rejecting coercion.

But what if those necessary tools are not sufficient?

So suggests Karen Stenner in The Authoritarian Dynamic, where she warns of the dangers of seeing intolerance as “a simple product of social learning” that will wane as the world’s cultures unlearn bad ideas. According to her scholarship, there will always be a subset of humans, across cultures and eras, whose deep discomfort with diversity predisposes them to support coercing others, as they reject and seek to undermine “any system that fails to promote oneness and sameness.”

Alas, these people are most likely to be activated in liberal democracies, where many will never feel entirely comfortable. And because their intolerance “springs from aberrant individual psychology,” rather than cultural norms, it is “bound to be more passionate and irrational, less predictable, less amenable to persuasion, and more aggravated than elevated by the cultural promotion of tolerance,” she argues. “Authoritarianism is a problem of and for libertarian, more than authoritarian, cultures. And intolerance is not a thing of the past, it is very much a thing of the future.”

If correct, this framework complicates the task of freedom’s champions.

It suggests we cannot merely push for advances in liberty, but must do so in a way that avoids a huge backlash, for what good is Radical Reconstruction if it ends in Jim Crow? What good is the Black Lives Matter movement if it ends in a “nationwide stop-and-frisk” presidency? What good is opening Europe’s borders to refugees if it brings neo-Nazis to power?

The point isn’t that all radical efforts to advance liberty, or even the ones that I’ve just mentioned, necessarily do more harm than good, but that backlashes matter and must be factored in. I prefer amnesty for undocumented immigrants, high levels of immigration, and generous refugee resettlement. My colleague David Frum argues that if liberals won’t enforce borders, voters will tap fascists to do it. If he is right, what rhetoric and policies ought the results-oriented libertarian champion, at least until he or she has persuaded more members of the public to change their position?

Barry Goldwater declared that “extremism in the defense of liberty is no vice,” but if authoritarians are triggered by change too extreme for their comfort, and if they reliably roll it back and then some, effective champions of liberty will avoid pushing too much or settling for too little.

That needn’t always mean compromising ends. How anti-authoritarians push might matter more than how much. What if while working to advance their values they sought out the manner that minimizes discomfort to people who fear difference or dynamism, so that their gains are more secure?

The liberty coalition often neglects that work, seeing it as beyond their purview. But the case for its importance is growing. Politicians with authoritarian tendencies have made gains in much of the West. Growing factions of American voters in the right-of-center and left-of-center coalitions are unapologetically illiberal––there is no partisan team whose victory will keep illiberals out of power. And the rise of Barack Obama and Donald Trump suggest that as people lose faith in politicians and establishment institutions, they do not gravitate toward small government libertarians, as one might hope, but to charismatic figures who promise they alone can fix a broken system.

Perhaps this authoritarian moment will pass.

It may be, however, that even more people with authoritarian predispositions are activated in coming years. Social science literature suggests that nothing activates those who value oneness and sameness more than a perception that a country’s leaders are failing and that its people are divided in their values. Today, faith in civic institutions is low, while racial and ethnic diversity are increasing, stoking anxieties that both the right and left exploit by doubling down on identity politics.

And the social web exposes everyone to the full range of diverging opinions and values.

Meanwhile, terrorism is an ineradicable risk that may increase as technology permits smaller numbers of people to effect greater harms. And authoritarians who exploit terrorism to bolster their power, or who are otherwise willing or able to use repressive measures, can now marshal surveillance tools so pervasive, intrusive, storable, and searchable that Big Brother would be envious.

For all that, the liberty coalition has surmounted seemingly insurmountable odds before. This fight does not begin with large swaths of America’s population in chattel slavery, or mired in the Depression as the Axis powers fortify their positions. It does not begin with Europe cleaved by an expansionist, totalitarian dictatorship with nuclear weapons, or with a wall running through Berlin.

Global liberty may not be at its apex, but neither is it close to its nadir. The liberty coalition has a fighting chance to reverse its losses and even to exceed former gains. But if so, it ought to hedge against the possibility that Karen Stenner is correct–– that a society reaps negative returns by exposing authoritarians to “more difference than they are predisposed to tolerate, and more democracy than they are innately equipped to handle.” As she puts it at the conclusion of her book, “democracy is most secure, and tolerance is maximized, when we design systems to accommodate how people actually are. Because some people will never live comfortably in a modern liberal democracy.”

In honor of the fortieth anniversary of our founding, the Cato Institute has organized a special online forum on the future of the free society. Join the conversation on Twitter with #Cato40, and follow the hashtag for updates.