Data Archiving, Disclosure and Due Diligence

"I have an ongoing campaign to improve standards of data archiving, disclosure and due diligence ... which are independent of any particular substantive points on paleoclimate studies. I have no idea why the “Hockey Team”, as they styled themselves, have elected to withhold data and methods from scrutiny; it’s an unwinnable position, but they’ve done so and I’ll continue to criticize them on this point." —Steve McIntyre

"The lack of documentation of data provenance, be it in Nature, Science, GRL, is shameful and I’ve done what I can to draw attention to the problem. It’s a disgrace." —Steve McIntyre (Post #14)

• All CA threads on Archiving

• Cicerone of NAS Acquiesces in Data Obstruction "Your performance in this matter has been shameful. The issues of climate change are important and neither you nor the National Academy of Sciences should be parties to the efforts of certain scientists to obstruct the archiving of important data. Regards, Steve McIntyre"

• NAS: Assuring the Integrity of Research Data "[Gerry North] said that the paleoclimate community was “shocked” to find themselves thrust into the limelight and “totally unprepared” for it. That seems only partly true - they seem quite prepared for awards from Scientific American and quite prepared for adulation. Indeed, not only were they prepared for it, they went so far as to issue press releases for many scientific studies - you don’t issue press releases if you aren’t seeking the limelight. What they seem not to have been “prepared” for is someone saying: if you communicate with the public, then you have responsibilities of disclosure and due diligence that exceed those applicable to discussion in seminar rooms."

• IPCC and Data Access "...when I sought to obtain supporting data for then unpublished articles, IPCC threatened to expel me as a reviewer." A truly bizarre episode.

• All CA threads on Disclosure and Due Diligence

• Some Thoughts on Disclosure and Due Diligence in Climate Science "I’ve found that scientists strongly resent any attempt to verify their results. One of the typical reactions is: don’t check our studies, do your own study. I don’t think that businesses like being checked either, but one of the preconditions of being allowed to operate is that they are checked. Many of the most highly paid professionals in our society — securities lawyers, auditors — earn much of their income simply by verifying other people’s results. Businesses developed checks and balances because other peoples’ money was involved, not because businessmen are more virtuous than academics." —SM

• "We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it." —Dr. Phil Jones, CRU, responding to a request for data from an independent researcher.

• Following are responses of Prof. Michael Mann to questions from Sen. Inhofe at the congressional hearing on the hockey-stick controversy:

30. Did IPCC carry out any independent programs to verify the calculations that you made in MBH98 or MBH99? If so, please provide copies of the reports resulting from such studies.

It is distinctly against the mission of the IPCC to “carry out independent programs”, so the premise of the question is false. However, the IPCC’s author team did engage in a lively interchanges about the quality and overall consistency of all of the papers as the chapter was drafted and revised in the course of review.

31. Did IPCC carry out any independent quality control on the data that you used in MBH98 and MBH99? If so, please provide copies of the reports resulting from such studies.

The IPCC doesn’t “carry out studies”, so the premise of the question is false. The IPCC instead depends that the normal scientific peer-review process, especially when done in a leading journal, has ensured an acceptable level of quality. In addition, the IPCC does check to see if any criticisms have been raised postreview in comments and response to the journal articles.

32. Did IPCC carry out any studies to validate the statistical procedures and methodologies used in MBH98 and MBH99? If so, please provide copies of the reports resulting from such studies.

The IPCC doesn’t “carry out studies”, so the premise of the question is false. Instead, as indicated above, the IPCC relies on earlier stages of review to cover such matters.

34. Has any organization other than IPCC conducted independent quality control on the data that you used in MBH98 and MBH99? If so, please provide copies of the reports resulting from such studies.

The IPCC doesn’t “carry out studies”, so the premise of the question is false. The data used by MBH98 (and MBH99) were produced by other researchers, not Mann and colleagues. It is thus not clear what kind of “independent quality control” is being referred to here. However, it is fair to say that each of these papers has been subject to rigorous peer review in a leading scientific journal, which is considered by scientists to be an independent quality control process. We are aware of no criticisms of the datasets in the peerreviewed scientific literature.

35. Has any organization other than IPCC carried out any studies to validate the statistical procedures and methodologies used in MBH98 and MBH99? If so, please provide copies of the reports resulting from such studies.

The IPCC doesn’t “carry out studies”, so the premise of the question is false. If the question were asked: Have other independent groups tested the methodology of Mann et al (1998) in a publication in the peerreviewed climate literature, the answer would be “yes”. I would refer the questioner to the following paper: Zorita, E., F. Gonzalez-Rouco, and S. Legutke, Testing the Mann et al. (1998) Approach to Paleoclimate Reconstructions in the Context in a 1000-Yr Control Simulation with the ECHO-G Coupled Climate Model, Journal of Climate, 16, 1378-1390, 2003. The paper arrives at the conclusion that the methodology of MBH98 performs well with networks of data comparable to those used by MBH98. [Mann's conclusion is strongly disputed by McIntyre & McKitrick (add ref)]

37. Did the peer reviewers for Nature in MBH98 carry out any independent quality control or validation studies? If so, please provide copies of such reports.

Neither I, nor authors of peer-reviewed journal articles in general, are made privy to the detailed analyses that peer reviewers may or may not have performed in the process of reviewing a manuscript. Authors only receive the comments that were selected to be made available to them by the reviewer and editor. This question is thus impossible to answer...

Your tax dollars at work!