Here, I just want to repeat my comment:
To start, yes I am generally unhappy with Steve McIntyre’s speculative tone that to me often implies the accusation of scientific misconduct or corruption of peer review where to me everything is in the scope of peer review (not only in climate science but in science generally). I would like to see his post-publication review submitted as comments to journals and I would appreciate if the journals would consider him as reviewer for reconstruction and proxy-papers.
Next a disclaimer: I am loosely associated with Euro2K. Therefore, I have to be careful in my next comments. So, if I appear evasive, I am sorry for that.
Another disclaimer: I am less structured in the following than I would like to be.
And a short comment on mentioning the Gergis et al. reconstruction. The Pages2K-Australasia-reconstruction is not the Gergis et al., the paper under review is only referred to for part of the method, not for the reconstruction. At least that’s what I read in the SI. Of course, it is close to it, but as Steve McIntyre correctly states any speculations on the relation of this reconstruction to the JClimate-paper have to wait until the latter is published.
I observed that there must have been a “interesting backstory” to the PAGES2K article, given its apparent rejection by Science and its already late appearance in IPCC schedule. I presume that the PAGES2K authors must have been dismayed at the rejection by Science, not least because they would then be very squeezed for time. I also presume that the IPCC chapter authors were concerned because they had presumed that acceptance of PAGES2K was a given. These points seem self-evident to me and are nothing more than human nature.
The acceptance of a paper is never “a given”, especially not with Science, Nature and Nature SomethingorOther. With hindsight, I can infer that the change of plan, i.e. Science to NatureGeo, required to postpone other papers.
I doubt they were dismayed but they weren’t happy either. Of course they were squeezed for time. But scientists tend to have deadlines all the time. So nothing new there? If I remember correctly, one of the IPCC chapter authors suggested before the submission deadline that it would be nice to have the paper ready for the IPCC but that one could also work with other published data if the Pages2K reconstructions were not published before the deadline.
That is: The Arctic-NorthAtlantic reconstruction by Hanhijärvi was accepted early February and could have been used. Kaufman et al (2009) would have been a choice as would have been Shi et al. (2012). The Guiot reconstruction for Europe could have been used as state of the art. Cook et al.’s reconstruction for East Asia was accepted November 2012 and could have been used. Wahl and Smerdon (2012) and Viau et al. (2012) would validly represent North America (Trouet et al. (2013) on the other hand was too late). For South America, we have the Neukom et al. data from 2011. Australasia and Antarctica would have been a problem, but so was Africa. Indeed the Africa2K paper by Nicholson et al. was accepted mid-February. For Antarctica, one could have just used the 200yr reconstruction by Schneider et al. (2006). Some of these choices would have changed the assessment but they would have been valid choices.
To distinguish between the methodological and the skeptical point of views: For the former the post-publication review of the PAGES2K will show whether there are holes which challenge the data and the conclusions; this also refers to the substantial holes comment by Jonathan Jones. For the skeptical point of view, I indeed think that the reconstructions and their discussion rather suit it.
On the substantial holes: Australasia. I don’t think that Steve McIntyre’s comments so far invalidate the reconstruction. South America is an update to Neukom et al. And, although I don’t like the formulation, these reconstructions are hypotheses based on the “best available” (selected on non-objective criteria) data.
So, as stated above, if there are strong indications of mistakes and errors, submit them as a comment or as a separate paper (as Hargreaves and Annan did in 2009), please. Put them on Arxiv at the same time. For example, why not put all your major objections to Marcott et al. in a comment and submit it first as a comment to Science and later to Climate of the Past or something similar, or put it as a separate paper on a “bladeless Holocene Hockey Stick” to any journal of choice. “Bladeless” is not stolen from Steve McIntyres recent post, but refers to my thoughts after the fray with Paul Matthews. I hope that Lewis (2013), Masters (2013) and some other papers (although of less quality than these two) should have proven that gate-keeping appears to be a smaller problem than it may have been. If you submit (and put the draft on Arxiv), some scientists are going to note and if (big if) you encounter gate keeping it’s more than easy to communicate it in a clear but less axe-in-the-face manner.
I remain somewhat puzzled as to the precise objectives of academic peer review as a form of due diligence. My experience is relatively limited, but it seems to me that reviewers too often try to impose their point of view, rather than letting authors have their say, while, at the same time, failing to ensure that the authors have provided a proper record that enables efficient replication. On the latter count, the PAGES2K authors have done a much better than average job according to standards in the field.
On Steve McIntyre’s comment on peer review in general. Yes, I think he’s right there. But the definition of peer-review is not a priori to try to replicate the results. Maybe Paul Matthews can comment whether mathematical peer review tries to follow each step in a paper and Jonathan Jones may comment on how this is done in the physical sub-fields he’s involved in. The task to ensure the possibility of replication lies with the author. The reviewer highlights gaps. At least that’s my impression. Should that be changed? No. The replication of results is part of the post-publication evaluation and failure to do so should be communicated in comments and possibly lead to retractions or corrections.
Put differently: The authors have to describe their methods so well that an informed reader can replicate their results with her prior knowledge and access to the data. Ideally the authors provide their code (well that should be requested). The reviewers have to check that the description allows the informed reader this replication.
So I am sure that the paper was subject to peer review as tense as average in science. Could it be more thorough. I would say peer-review can always be more detailed.
However, I would be surprised if the peer reviewers, either for Science or Nature, did any serious due diligence. PAGES2K presented reconstructions for 7 continents, involving a variety of different proxies, and using several different methods, at least of one of which was unpublished at the time of submission of the article. This is a lot of material and different issues to cover in one article on a short deadline.
I understand these concerns. However, I would interpret the time period between the initial submission to Science and the subsequent submission to Nature Geoscience as signaling that there was a lot of work to do for the authors to deal with the initial comments. Assuming (with a at least 50 percent chance of being wrong) that these initial comments and the made changes were highlighted in the second submission, I think the review process was as thorough as I expect it to be for these journals. Does that mean it was as thorough as to ensure perfection of the published manuscript. That’s not the point of peer review. And with respect to unpublished results: The reviewers are encouraged to request all the material they need to provide a thorough review. Further, as I already wrote, the work involved in revising this paper has possibly lead the author’s to postpone other work (I know of one paper).
One reasonable review response might well have been that the authors should publish their regional reconstructions in specialist journals. And that the authors should publish a detailed analysis of the methodologies in a specialist journal. It seems entirely possible that Science might have taken that position in rejecting the article.
To add speculation to speculation. I assume that the consortium asked Science whether they would be interested before formal submission. That’s common and even encouraged by the glamour-journals. And I can say that the idea was, in principle, to publish the reconstructions in specialist journals and to submit this synthesis paper to one of the “Letter”-journals.
Without the looming IPCC deadline and the prominent use of PAGES2K results by IPCC, I believe that it is entirely reasonable that Nature would have taken a similar position (to my interpretation of Science) and told the authors to split the article up into manageable review pieces. Do I believe that Nature recognized the need for very rapid acceptance and selected reviewers who also recognized the problem? Yes.
Again speculation to speculation. I think the synthesis provided by the PAGES2K consortium perfectly fits the scope of a progress-article in Nature Geoscience. McIntyre’s last question has to be answered with yes, but do I think that prevented a thorough peer review? No. See list of possible alternatives above.
Plus: What Richard Betts says & anyway, I am confident that at least Fredrik, Eduardo, Jan, Jason and Johannes have worked as internal quality control for proxies and methods.
Steve McIntyre replied to my comment. I don’t see why I should reply to that, but just for completeness sake: I think he raises interesting points. That should suffice. The next post will continue this story.