Even if you generally liked how Justice Anthony M. Kennedy voted, you had to scratch your head at how he got there. I should know; during his time on the bench, he agreed with the Cato Institute more than any other justice did. Cato was the only organization in the country, for example, to file briefs supporting the side that got Kennedy’s vote in virtually all the big cases: District of Columbia v. Heller Second Amendment), Citizens United vs. Federal Election Commission campaign finance), Shelby County v. Holder voting rights), United States v. Windsor (Defense of Marriage Act), National Federation of Independent Business v. Sebelius Obamacare), Obergefell v. Hodges (same-sex marriage) and this term’s Masterpiece Cakeshop v. Colorado Civil Rights Commission free exercise clause ).

But that didn’t make him any less frustrating to libertarians — at least those of us who care about the rule of law rather than simply achieving what we consider to be good policy results.

Take his celebrated opinion in Obergefell in 2015. What should have been an easy case about the propriety of certain marriage-licensing schemes under an equal protection clause that says the government can’t treat people differently for no good reason, instead became yet another opportunity to wax poetic about the meaning of life. The rule of Obergefell seems to be that you take a scoop of due process and a cup of equal protection, wrap them in some dignity, and then enjoy the waves of adulation.

That’s not law. For one thing, there’s no fundamental right to the state recognition of marriage, which is a kind of government benefit. To put it in the context of injustices perpetrated against gay people, marriage isn’t like the right to have sex with a consenting partner (which Kennedy upheld in 2003’s Lawrence v. Texas ). After all, a logical extension of Kennedy’s understanding of the fundamental right to marriage is that states are prohibited from getting out of the marriage business altogether. That can’t be right.

Moreover, a constant theme in Kennedy’s gay rights jurisprudence — he’ll be known as the “gay justice” until someone who is openly gay is seated on the court — is the ignoble motivation behind the laws at issue. According to Kennedy, such laws are illegitimate because they represent character flaws in those who support them. Contrast them with his silent acquiescence to the federal regulation of medical marijuana use in Gonzales v. Raich  (2005) and his opinion in National Treasury Employees Union v. Von Raab  (1989), in which the court — over dissents written by Justices Antonin Scalia and Thurgood Marshall! — upheld the suspicionless drug testing of government employees.

The only way to explain these votes is that Kennedy doesn’t see any dignity to drugs, so unlike the LGBTQ community, their users merit little constitutional protection. A faithful constitutionalist wouldn’t accept oppressive laws even if there’s a “noble purpose,” but Kennedy has a sense of noblesse oblige.

This isn’t just about sex and drugs. Kennedy’s jurisprudence on abortion is a mess. (An “undue burden,” goes the joke about the standard he set 26 years ago in Planned Parenthood of Southeastern Pennsylvania v. Casey, is anything that gives Kennedy a headache.) And his decisions in race cases were infuriatingly hand-wringing. He accepts, for example, that encouraging racial diversity — especially in education — is constitutionally legitimate but, until 2016’s quixotic Fisher v. University of Texas,  had never voted to uphold a policy aiming at that goal.

Only on the First Amendment has he been a clear and consistent interpreter of the Constitution. Indeed, in campaign-finance cases, it is Chief Justice John G. Roberts Jr. who has long been the minimalistic swing vote, not Kennedy — who authored Citizens United as a full-throated rebuke of a 1990 case (Austin v. Michigan Chamber of Commerce) in which he had dissented.

The problem with Kennedy isn’t that he’s a squishy moderate who sometimes agreed with progressives and sometimes with conservatives. Nor that he was a pragmatic opportunist like his predecessor as the court’s swing vote, Justice Sandra Day O’Connor. It’s that, even if Kennedy had a particular view of constitutional interpretation — some doubt it because he seemed to see each case as unique — his paradigm was indecipherable to the mere mortal.

Decisions that come from such a special access to truth are incompatible with the rule of law, which values predictability and transparency. If a black box made all of society’s key decisions, it wouldn’t be legitimate governance even if the results are attractive.

Regardless of how convincing anyone’s explanation of Kennedy’s jurisprudence may be, if the general perception is that he decided cases in some inscrutable manner, whether based on policy preferences or some unrecognizable legal theory, he’s no better than that black box.

To put it another way, Kennedy was like a prophet who struggled mightily to convey the will of the Founding gods. I appreciated that effort, really — particularly given that his revelations so often matched my preferences — but that sort of prophecy can only result in a decline in the congregation’s faith in the law.