Oil prices seem to be in free-fall. After averaging a staggering $137 a barrel over the first week of July, they were down to $109 a barrel over the final week of August.

Where are prices going next? Who knows? Bearish talk about bubbles bursting and bullish talk about peak oil disguise the fact that the future direction of oil prices is unknown and unknowable. Neither investors nor politicians ought to be betting the economic house on any particular vision of “our energy future.”

The fundamental reason for the difficulty associated with forecasting future oil prices is the fact that both demand and supply are relatively inelastic over the short term. That means rather small changes in either can have very large price effects. Hence, those who wish to forecast oil prices are forced to forecast weather patterns, labor relations, gross domestic product (GDP) reports, demographic trends, civil unrest and technological change in all sorts of disparate economic sectors.

Long-run forecasts are no easier to execute. Professor Vaclav Smil of the University of Manitoba has cataloged the vast record of energy forecasts offered by academics, corporations, consultants, trade associations, government agencies, “blue ribbon” commissions, policy activists and “futurists” of all stripes over the past 100 years and finds a “a manifest record of failure.” There is simply no reason to believe that mere mortals can foretell oil prices or petroleum market shares in the future, absent some sort of time machine.

A recent analysis of world crude oil prices by Professor James Hamilton of the University of California at San Diego reinforces Smil’s point. In a paper published by the University of California Energy Institute, Hamilton looked at data from the first quarter of 1970 through the first quarter of 2008 and asked the question, “How predictable statistically is the change in the real price of oil over this period?” It turns out that:

  • Neither nominal U.S. interest rates nor real U.S. GDP growth rates can predict oil price movements.
  • Although prices increased by 172% (logarithmically) over the sample period (an average of 1.12% per quarter), Hamilton could not reject the null hypothesis that there was no trend in the data.
  • The data is most consistent with the observation that oil prices move akin to “a random walk without drift.”
  • The best predictor of future oil prices is the present oil price.

If the best predictor of future prices is present price, how then do we account for the extreme volatility in the record? Simple: Present price may be the best predictor, but it is a lousy predictor nonetheless. Hamilton finds that the standard deviation in oil prices from quarter-to-quarter was 15.28%. Hence, if we start a quarter with $115 oil, prices in the next quarter could average between $85 and $156 per barrel. In a year, they could range between $62 and $212. In four years, they might be anywhere between $34 and $391!

Of course, if oil prices were to rise — or decline — to the upper or lower boundary of our “random walk,” market analysts and policy pundits would likely wet their pants over “the end of the oil age” (in the case of the former) or the end of OPEC (in the case of the latter). While trends in those cases would be perfectly consistent with either narrative, they would also be perfectly consistent with a random walk … and no real trend.

There is an insatiable market demand for oil price forecasts and an equally insatiable political demand for mega-explanations about oil prices. But unless the oil market changes pretty radically, neither demand can be met by well-informed oil market analysts.