If we could have only gotten “yes, affirmative action,” or “no, affirmative action,” as the outcomes in Students for Fair Admissions v. Harvard, which the U.S. Supreme Court decided last week, you should have pulled for the decision the Court delivered: public colleges, or privates that receive federal funding, cannot use race as a factor in admissions. Other things equal, we want people treated as individuals, not as members of a race, as director of Cato’s Center for Constitutional Studies, Anastasia Boden, wrote soon after the decision was announced.

But Students for Fair Admissions did not present a simple choice, at least as policy goes, nor are all things equal. The decision combined two cases: One against the University of North Carolina – a state school – the other against private Harvard University. Those schools, by virtue of one being a government institution and the other not, should not be treated the same. It is much more consistent with a free society, as well as diverse views about what constitutes a fair admissions policy, to allow private institutions to make their own decisions than to have government, especially Washington, impose uniform rules on all.

Of course, higher education involves a great deal of federal money, especially in the form of grant and loan aid to students, as well as research. That gives the federal government a large financial presence in the ivory tower. But even with that, the default should be autonomy for private institutions – there is more freedom with funding and no strings than with strings attached.

Affirmative action is, crucially, different from past racial discrimination, in which people of specific races, especially African Americans, were intentionally kept out of institutions because of their race, as opposed to schools seeking to be more inclusive of all racial groups. Indeed, as the dissent in Students for Fair Admissions emphasized, affirmative action is, at least in part, intended to more fully incorporate African Americans into higher education specifically because they suffered centuries of rank, race-based discrimination and oppression.

One can disagree with that reparative reasoning, yet still see how decent people can support it.

The other major argument for allowing colleges to take race into account in admissions is that there is educational and social value to a racially diverse student body. This might not be empirically proven – little involving human beings is – but it is easy to conceive how a student’s education would be enriched by learning and living with students of diverse backgrounds. And research suggests that when people from different groups come together, it helps to build bridges among them.

There are, simply, reasonable grounds for assembling racially diverse student bodies. But if so, why only allow leeway for private institutions? Why not public, too?

Because public colleges and universities are government institutions, receiving funding directly from state taxpayers and answerable to the state. And government actions are ultimately done at the point of a gun – a legal monopoly on force that can result in jailing or worse for those who do not comply. Hence, government was the primary enforcer of past racial injustice, from slavery, to Jim Crow, to discriminatory allocation of housing assistance. That power makes it inherently more dangerous to let government consider race in allocation of resources than private institutions. As former Center for Constitutional Studies Director Ilya Shapiro wrote in Cato’s amicus brief asking the Court to join Students for Fair Admissions’ suit against Harvard with its suit against UNC:

Racial preferences by private universities certainly have negative effects on individuals too. But it is one thing to be viewed through a racial lens by a private institution or fellow citizen, quite another—and substantially more harmful—for a polity to reduce people to racial identifiers. Government racial classifications demean all citizens. Grutter v. Bollinger, 539 U.S. 306, 353 (2003) (Thomas, J., concurring in part and dissenting in part).

The majority in Students for Fair Admissions acknowledged that there might be reasonable grounds for universities to consider race in constructing classes. A footnote saying that military academies were not part of the decision mentions that a federal government brief argued that at such institutions “race-based admission programs further compelling interests.” The footnote says that the majority “does not address the issue, in light of the potentially distinct interests that military academies may present,” seemingly conceding that some schools might need to balance students’ races.

Justice Sotomayor picked up on this in her dissent, noting that many schools have military training programs, and other institutions also have “distinct interests”:

To the extent the Court suggests national security interests are “distinct,” those interests cannot explain the Court’s narrow exemption, as national security interests are also implicated at civilian universities. See infra, at 64–65. The Court also attempts to justify its carveout based on the fact that “[n]o military academy is a party to these cases.” Ante, at 22, n. 4. Yet the same can be said of many other institutions that are not parties here, including the religious universities supporting respondents, which the Court does not similarly exempt from its sweeping opinion. See Brief for Georgetown University et al. as Amici Curiae 18–29 (Georgetown Brief ) (Catholic colleges and universities noting that they rely on the use of race in their holistic admissions to further not just their academic goals, but also their religious missions); see also Harvard II, 980 F. 3d, at 187, n. 24 (“[S]chools that consider race are diverse on numerous dimensions, including in terms of religious affiliation, location, size, and courses of study offered”).

Recognizing that race can matter, the majority identified an avenue by which schools can consider it: if an applicant writes an admissions essay that mentions how their race has impacted them. “[N]othing in this opinion should be construed as prohibiting universities from considering an applicant’s discussion of how race affected his or her life, be it through discrimination, inspiration, or otherwise,” says the decision.

Neither federal student aid nor research funding require the federal government to take a position on the “right” admissions policy. Student aid is directed to institutions based on millions of recipients making choices about the colleges they wish to attend. They can freely decide, using their own, diverse weightings of competing values, including academic performance and racial inclusivity, which institutions have acceptable policies, and which do not. Research funding, meanwhile, should be aimed at a particular end – gaining new knowledge in a specific area – not supporting an institution.

Absent compelling evidence that their policies are intended to exclude specific racial groups, as opposed to maximize inclusivity, private institutions accepting federal funds should be able to set their own admissions policies. This might require amending Title VI of the Civil Rights Act, which is the Court’s basis for applying its affirmative action prohibition to private colleges.

There are, alas, no easy answers to questions of race because of the centuries-long, stark clash between our ideals of colorblind equality and what our government and society have actually done. When we have such wicked problems, it is best to let diverse people make decisions for themselves, weighing their own valuations of goods including colorblindness, racial diversity, repairing historical wrongs, and more.

It is best to embrace what should always be our default: liberty.