Michael Phelps, the aquatic icon who won eight gold medals at the 2008 Olympics, has violated the law. When a photograph of him smoking a bongful of marijuana was published, he admitted the crime. The same crime for which the better part of a million people were arrested last year.

Shouldn’t Phelps be charged? Along with President Obama and his two predecessors, all of whom, it seems, used illegal drugs? If not, perhaps it is time to have a serious debate about the drug laws.

Of course, Michael Phelps immediately apologized for his poor judgment. Attention turned to his sponsors, since their contracts include the usual moral clauses, which protect their investment in celebrities who behave foolishly, if not actually immorally. Happily for Phelps’s bank account, some of his big-money backers, including Speedo, Hilton, and Omega, accepted his apology. Subway and Visa haven’t been talking, but don’t look like they are going to jump. Kellogg’s, so far in the minority, announced it would drop Phelps.

But if marijuana use is so horrid as to warrant criminalization, why are we wasting time discussing whether Phelps will be able to keep his endorsement deals? Shouldn’t he be prosecuted—just like millions of other Americans, whose lives have been ruined by criminal convictions for smoking pot?

In 2007, 872,721 Americans were arrested for marijuana violations, 775,138 of them for possession. Some number of the latter undoubtedly were caught growing or selling and were charged with lesser offenses, but, in any case, hundreds of thousands of Americans ended up in jail for doing precisely what Michael Phelps did: lighting up. Roughly three-quarters of those arrested for marijuana offenses were, like Phelps, under 30. With most of their lives ahead of them, they face the greatest harm from prosecution under the drug laws.

So why shouldn’t Phelps go to jail?

To ask the question is to answer it. While smoking pot may be a stupid thing to do for many reasons—risking adverse health effects, endangering endorsements, undermining Phelps’s status as a celebrity role model—he hurt no one but himself. He could have been photographed while drunk and stumbling out of a party, and it would have been no different. Bad press and angry sponsors would have forced an abject apology, and everyone would have moved on. Just like with his marijuana hit.

Of course, advocates of prohibition argue that illicit drugs are different. And so they are—mostly because their use is illegal, a situation that creates the most serious problems usually associated with drug use.

The arguments are old but clear. Whatever the law might say, the people have voted with their lungs: 95 million Americans over the age of 21 have smoked pot, 20 million have smoked in the last year, and 11 million use the drug regularly. It’s hard to believe that all of them, almost one-third of the U.S. population, are criminals who deserve jail time.

Moreover, the violence associated with drugs is principally from prohibition rather than use. Drunks are far more likely to commit (and be victims of) violent crimes than are users of marijuana. Prohibition-era Chicago offered a dramatic lesson in the impact of banning a widely used drug. That city’s violent era is being played out on a larger scale in Colombia and Mexico, where urban and rural communities have been overwhelmed with drug-gang violence.

The health arguments remain disputed, but the basic question is whether we live in a free society in which people can choose to engage in risky behavior. Cigarette smokers, hang gliders, and rock climbers all take risks that many others view as unacceptable. That’s no reason for arresting them.

And it’s pretty hard to argue that marijuana use will prevent Phelps from being productive. Most all of us probably remember pothead classmates who ended up wildly successful in their chosen careers. Will some people use to excess? Yes, just as some people drink too much, gamble too much, spend too much, and act irresponsibly in a multitude of other ways. Criminal law is not the answer.

Is Michael Phelps likely to go to jail? No, and for good reason. But for the same reason, the rest of us should not be arrested for smoking pot, either. Whether marijuana use is good or bad is not the issue. Short of engaging in behavior that directly threatens others, people should be left alone. That’s what a society grounded in individual liberty is—or at least should be—all about.