… at least, that’s what one might think about the state of the literature today after reexamining a paper by Isabelle Chuine et al. that was published a couple of years ago in the journal Nature (subscr. required).

Chuine et al. claimed to have developed a method for estimating summer temperatures in the French wine region of Burgundy from 1370 to the present based on the dates that grapes were harvested. Using this method, the authors asserted that the summer of 2003 was the warmest summer in Burgundy since 1370. The study was offered up as yet one more piece of evidence that global warming is running amuck.

But not so fast — it turns out that the estimates offered by Chuine et al. have absolutely no relation with observed temperatures and that no one ever bothered to check whether their estimates matched hard data when the two coincided. In a forthcoming paper for Theoretical and Applied Climatology, Douglas Keenan tears the Chuine paper limb from limb.

More importantly, Keenan calls attention to the overall shoddiness of the scientific press today and the researchers publishing therein:

What is important here is not the truth or falsity of the assertion of Chuine et al. about Burgundy temperatures. Rather, what is important is that a paper on what is arguably the world’s most important scientific topic (global warming) was published in the world’s most prestigious scientific journal with essentially no checking of the work prior to publication.

Moreover — and crucially — this lack of checking is not the result of some fluke failures in the publication process. Rather, it is common for researchers to submit papers without supporting data, and it is frequent that peer reviewers do not have the requisite mathematical or statistical skills needed to check the work (medical sciences excepted). In other words, the publication of the work of Chuine et al. was due to systemic problems in the scientific publication process.

The systemic nature of the problems indicates that there might be many other scientific papers that, like the paper of Chuine et al., were inappropriately published. Indeed, that is true and I could list numerous examples. The only thing really unusual about the paper of Chuine et al. is that the main problem with it is understandable for people without specialist scientific training. Actually, that is why I decided to publish about it. In many cases of incorrect research the authors will try to hide behind an obfuscating smokescreen of complexity and sophistry. That is not very feasible for Chuine et al. (though the authors did try).

Finally, it is worth noting that Chuine et al. had the data; so they must have known that their conclusions were unfounded. In other words, there is prima facie evidence of scientific fraud. What will happen to the researchers as a result of this? Probably nothing. That is another systemic problem with the scientific publication process.

Unfortunately, few enviro-beat reporters take the time to critically examine the avalanche of papers crossing their desk claiming this, that, or the other. Fact-checking rarely if ever occurs. The Globe and Mail, for instance, was absolutely breathless about Chuine’s findings. After all, what a topical hook: global warming will screw up your pinot noir!

The lesson here is that you can’t assume that anything in the scientific literature has ever been given even a cursory critical review prior to publication. Peer-review means nothing. Swallow this stuff at your own risk.