Has the "Hockey Stick" Been Disproven? Part 3 - North and Wegman Reports

Hockey Sticks Featured in the North Report

In two previous posts (here and here), I described the challenges by McIntyre and McKitrick (M&M) to the initial two hockey stick papers published by Mann, Bradley and Hughes (MBH98 and MBH99). In these posts I summarized M&M's multiple critiques of the MBH hockey stick papers, essentially that the "hockey stick" is an artifact of flaws in the MBH statistical method and an over reliance on one set of North American tree ring proxies. However, multiple peer-reviewed papers following M&M's criticism generally found that M&M's criticisms lacked merit and vindicated the MBH hockey stick reconstruction. To summarize:

1. Biases associated with MBH's statistical method were small and contributed very little to the shape of the MBH hockey stick reconstruction. Biases associated with M&M's statistical method went in the opposite direction, and M&M exaggerated the effect of statistical bias on the shape of the hockey stick.

2. The "hockey stick" emerges from the proxy data regardless of statistical method. It emerges using M&M's method provided you don't censor the data as well as from MBH's method. It emerges if you don't use PCA at all.

3. Other studies using different proxies and methods also arrived at similar results to MBH98/99, essentially replicating the MBH hockey stick (like Moberg 2005).

4. A comprehensive analysis by Wahl and Ammann showed the MBH hockey stick reconstruction to be robust and M&M's criticisms to be "without statistical and climatological merit."

However, by 2005 the "hockey stick" had become intensely politicized, with several Republican leaders uncritically accepting the M&M criticisms of the hockey stick and believing that Mann and his colleagues were guilty of flawed analysis, scientific misconduct, or even fraud. Consequently, Sherwood Boehlert, chairman of the U.S. House of Representatives Committee on Science, requested an independent analysis from the National Academy of Sciences. The resulting report has been referred to as the NAS Report or the North Report.[1]

North Report

The NAS report generally affirmed the findings of MBH98/99. Gerald North said, “We roughly agree with the substance of their findings," though he indicated that the committee had less confidence in the results for time periods prior to A.D. 1600. It's worth quoting the report here.

It can be said with a high level of confidence that global mean surface temperature was higher during the last few decades of the 20th century than during any comparable period during the preceding four centuries. This statement is justified by the consistency of the evidence from a wide variety of geographically diverse proxies.

Less confidence can be placed in large-scale surface temperature reconstructions for the period from A.D. 900 to 1600. Presently available proxy evidence indicates that temperatures at many, but not all, individual locations were higher during the past 25 years than during any period of comparable length since A.D. 900. The uncertainties associated with reconstructing hemispheric mean or global mean temperatures from these data increase substantially backward in time through this period and are not yet fully quantified.

The results here largely agree with the MBH98/99 studies. For instance, the graph published in MBH99 shows the confidence intervals being much wider prior to A.D. 1600, indicating that MBH have less confidence in their findings prior to A.D. 1600 as well.

In a press release following the publication of the NAS report, Mann was quoted as approving of the findings of the NAS report. "Mann says that he is 'very happy' with the committee’s findings, and agrees with the core assertion that more must be done to reduce uncertainties in earlier periods. 'We have very little long-term information on the Southern Hemisphere and large parts of the ocean,' he says." And in fact, more has been done since 1999 to reduce uncertainties in earlier periods. Global reconstructions with more extensive proxy networks have continued to replicate the MBH hockey stick and provide even more confidence that the last decades of the 20th century were the warmest of the last 2000 years, and warming has continued to accelerate into the 21st century. 

Wegman Report

However, Joe Barton of the US. House Committee on Energy and Commerce decided to launch his own investigation, and he took a more ad hominem approach. To me Boehlert's approach had been at least understandable. Politicians are not scientists and can't be expected to wade through the literature and evaluate it on their own. The North report provides a valuable service in that regard. It validated MBH98/99 and also cautioned against putting too much confidence in the early centuries of the reconstruction. But Barton essentially attempted to intrude into the scientific process, interfere with academic freedoms and even the personal lives of Mann, Bradley and Hughes. He and Whitfield sent several questions to the three authors, including requests to supply their CVs, to justify their financial support and work for which they received federal grants, and to provide a list of locations of data archives. They asked MBH to explain why they didn't supply their proprietary code (which they were under no obligation to provide), an accounting of requests for data they had received, and an explanation for "alleged errors" found by M&M, including a request for "the R2 statistic for the temperature reconstruction." And they were asked to provide a detailed explanation of their activities for the IPCC and the Third Assessment Report. Boehlert called this a "misguided and illegitimate investigation," and he was not alone. Many scientists came forward saying that this was an unnecessary intrusion of politics into the scientific process and was designed to intimidate scientists from working in policy-relevant areas of scientific inquiry.

Barton also reached out to Edward Wegman and asked him to write yet another evaluation of the MBH hockey stick, and it appears that Barton hand-picked Wegman to produce a paper that would reinforce M&Ms' critiques of MBH98/99.  The resulting report, called the Ad Hoc Committee Report on the 'Hockey Stick' Global Climate Reconstruction (hereafter the Wegman Report) was a bit of an outlier. It was not a peer-reviewed paper, and it came to conclusions that had already been shown to be deficient in the peer-reviewed literature. None of the authors chosen by Wegman were climate scientists, paleoclimate experts, or dendrochronologists; Wegman himself was a statistics professor at George Mason University. 

The Wegman Report had two main parts. After an introduction, background, and a highly selective survey of the paleoclimate literature of the time, the report examined the statistical methods, in particular, the principle component (PC) analyses used by MBH and M&M. Unsurprisingly, Wegman essentially agreed with M&M's criticisms of MBH, though without quantifying whether the reported flaws in MBH98/99 had a significant impact on the reconstruction. Following this, the report supplied the results of a "social network analysis" designed to show that the scientists associated with the MBH hockey stick reconstruction were all closely connected with Michael Mann and somewhat isolated from the broader statistical community. The intent of this section appears to be to infer that paleoclimate scientists held allegiances to Michael Mann that caused them to support what Wegman believed was a faulty methodology and the conclusions that arise from it.

Statistical Methods

The evaluation of statistical methodology by the Wegman report offers very little by way of new information or critique beyond what was already in M&M's papers. In fact, it appears that they just repeated M&M's claims without any careful analysis. The Wahl and Amman 2006 (WA06) paper was known by the authors of the Wegman report, but there is no rebuttal of or correction to it. In fact, in a footnote, the report simply says that M&M's work was "critiqued by Wahl and Ammann (2006) and the Wahl et al. (2006) based on the lack of statistical skill of their paleoclimate temperature reconstruction. Thus these critiques of the MM05a and MM05b work are not to the point." This is a baffling dismissal of WA06, which had essentially refuted every single point of substance raised by M&M. WA06 concluded that

the Mann et al. reconstruction is robust against the proxy-based criticisms addressed. In particular, reconstructed hemispheric temperatures are demonstrated to be largely unaffected by the use or non-use of PCs to summarize proxy evidence from the data-rich North American region. When proxy PCs are employed, neither the time period used to center the data before PC calculation nor the way the PC calculations are performed significantly affects the results, as long as the full extent of the climate information actually in the proxy data is represented by the PC time series.

They further found that M&M's critiques were "without statistical and climatological merit." The findings of WA06 were also highlighted in Mann's response to the questions asked of him by Barton, so substantive interaction with the findings of WA06 should have been an essential component of the Wegman report. But when asked about the omission of any substantive interaction with WA06, Wegman answered with

It is our understanding that when using the same proxies as and the same methodology as MM, Wahl and Ammann essentially reproduce the MM curves. Thus, far from disproving the MM work, they reinforce the MM work. The debate then is over the proxies and the exact algorithms as it always has been.

This is a clear misreading of WA06. Sure, WA06 was able to reproduce the MM curves, but they also demonstrated that M&M's reconstruction (Scenario 1 below) showed a validation failure over the crucial time frame to their argumet. They also showed that the hockey stick in MBH98 was robust to statistical method. WA06 in no sense "reinforced" M&M's work. Wegman essentially dismissed and ignored the analysis of WA06, which is a clear and glaring oversight on the part of Wegman's team.

 

Social Network Analysis

The next section which reports to conduct a "social network analysis" of the scientists involved in paleoclimate studies. This is at least superficially more interesting, but here the ad hominem nature of  the Ad Hoc Committee set up by Barton becomes even more obvious. It's also thoroughly speculative. This is clear right from the beginning:

if there is a tight relationship among the authors and there are not a large number of individuals engaged in a particular topic area, then one may suspect that the peer review process does not fully vet papers before they are published. Indeed, a common practice among associate editors for scholarly journals is to look in the list of references for a submitted paper to see who else is writing in a given area and thus who might legitimately be called on to provide knowledgeable peer review. Of course, if a given discipline area is small and the authors in the area are tightly coupled, then this process is likely to turn up very sympathetic referees. These referees may have coauthored other papers with a given author. They may believe they know that author’s other writings well enough that errors can continue to propagate and indeed be reinforced.  

Here Wegman speculates that if there aren't that many paleoclimate scientists, and if they are "tightly coupled," then the peer-review process could become contaminated by "sympathetic referees." And in Wegman's mind, this undermines "how confident one can be in the peer review process."

Generally speaking, scientific journals do not publish the names of the referees of their papers. In a single-blind review process, referees know the authors' names but the referees themselves are anonymous. In double-blind review, the referees aren't given the names of the authors. It's unclear to me how papers published in journals following a double-blind review process could be affected by these social networks. At the time MBH98 was published, Nature followed conventional single blind peer-review, so Wegman would not have had access to the names of the referees for MBH98, nor would he be able to read how critical their comments may have been. Wegman has no direct evidence that there were "sympathetic referees" allowing the propagation and reinforcement of errors in MBH98 or any of the subsequent papers that validated and replicated the MBH hockey stick.

Undaunted by lack of direct evidence, Wegman decided to speculate based on his reconstruction of the "social networks" of authors publishing paleoclimate research, assuming that the more these authors collaborate the more likely the peer-review process would propagate and perpetuate errors. I have a few thoughts about this:

1. Wegman's diagrams are what they are, but the implications he makes from them are purely speculative. Wegman doesn't have access to the information he would need to conclude that "sympathetic referees" were propagating and reinforcing errors. 
2. The social networks he constructed are very Mann-centric, but at the time MBH98 was published, Mann had just earned his PhD. He was essentially the new kid on the block with paleoclimate research. Many of the social networks he describes were likely formed mostly after the publication of MBH98/99. It's very strange to imagine that networks that developed after the publication MBH98/99 might have caused the referees for MBH98 and MBH99 to be overly sympathetic.
3. Wegman asserted that other proxy reconstructions were "not as independent as one might guess," but the peer-reviewed literature already acknowledged that various reconstructions use some of the same proxies, and so these paleo temperature reconstructions are not fully independent. 
4. We have evidence that these authors were not just reinforcing each other's conclusions. Briffa in particular shows that he was concerned about being careful not to say more than what could be said from the data. And the hacked CRU emails (which were admittedly made public after the Wegman report) reinforce this. Email dialog between many of these authors show that they were concerned to get the data correct, understand discrepancies between the reconstructions, and not just promote the MBH hockey stick.
5. Wegman infers from the fact that Mann had co-authored papers with 42 other people that all of these co-authors "have close ties to Dr. Mann." But it would seem to me that it would be difficult to maintain "close ties" with 42 people. Wegman is simply speculating.
6. The claim that Mann's associates were isolated from the larger statistical community is without sound foundation. Mann was a member of the AMS Committee on Probability and Statistics from 2003-2005, where he worked with other scientists and statisticians. Doug Nychka, the head of the National Center for Atmospheric Research Geophysical Statistics Project was consulted for WA06. And von Storch was a co-author with Francis Zwiers of Statistical Analysis in Climate Research.
7. Wegman and his co-authors do appear to be isolated from the larger paleoclimate community, and they didn't appear to be aware of statistical communities associated with meteorology and atmospheric research.
8. There were contrarian papers published in otherwise reputable journals that negatively evaluated the MBH hockey stick; these papers passed peer-review. Willie Soon and Sallie Baliunas collaborated on a paper on the Medieval Warm Period, and McIntyre and McKittrick collaborated on papers evaluating the MBH hockey stick. Presumably the referees for these papers were not so sympathetic to Mann that they rejected these papers for publication, despite the flaws these contrarian papers contained.
9. If we were to accept the speculation that Wegman makes from these social networks, the contrarian social network was much, much smaller. If the size of Mann's social network makes it more likely that reviewers would propagate errors, then M&M's much smaller social network would make it even more likely, especially given confirming evidence from Energy & Environment that this journal gave preferential treatment to contrarian opinions on climate and even rushed M&M's paper to publication for political purposes.
10. Now in 2023, dozens of papers have been published replicating the hockey stick (we're at 62 and counting). The list of authors publishing in the field has also considerably increased. Wegman's opinions about paleoclimate research are even less credible now than it was when the Wegman report was published. 

In my opinion this section of the Wegman report is off-topic speculation and opinion based not just on insufficient data but the wrong kind of data. This had no business being in the report, both because it's largely ad hominem and because it has literally no bearing on the validity of MBH98/99. Even Wegman admitted that this was a "hypothesis" that "should be taken with a grain of salt." If so, it should never have been included in the report. The Wegman report failed to interact with most of the relevant material critical of M&M's work, and it failed to demonstrate flaws in any of the other reconstructions that validated and/or replicated the MBH hockey stick. In the absence of any compelling evidence that these papers were flawed, it would seem Wegman simply decided to speculate that MBH were wrong because of a kind of group think arising from supposed allegiances to Michael Mann.

Towards the end of this section of the Wegman report, Wegman posted a graph of various reconstructions, which he says are "taken from" D’Arrigo et al. (2006). The graph is actually an adaptation of a graph in that paper with the instrumental record removed.

There are multiple claims that Wegman makes from this graph which show that he didn't actually do a thorough evaluation of D’Arrigo et al. (2006).[3] I'll quote them and respond to them in turn:

1. "There are variations in the temperature reconstruction indicating the fundamental uncertainty in the reconstruction process." Actually the graphs did not contain uncertainty envelopes and they do not all reconstruct the same thing. Some are land only while others are land and ocean. Some are NH, while others are extratropical NH. Some are annual reconstructions while others are biased towards summer temperatures. The differences between the graphs are partly due to uncertainties in the proxy data and partly due to differences in what is being reconstructed.
2. "Essentially all agree that there was a medieval warm period centered on AD 1000 and a little ice age from at least 1600 to 1850." Well, MBH99 and Jones 98 only go back to AD 1000, so it would be impossible to say that the MWP was centered on AD 1000 in these reconstructions.
3. "There is consensus in these reconstructions that the global average temperature has risen over the last 400 years." It looks to me like there was cooling going into the early 19th century in pretty much all these reconstructions.
4. "Both Esper et al. (2002) and Moberg et al. (2005) indicate that current global temperatures are not warmer that the medieval warm period." That's not correct. None of these graphs show "current" temperatures because Wegman removed the current temperatures from the graphs. And the warmest decade in Esper 2002[4] was 1950-1959. Moberg 2005[5] doesn't go as far into the 20th century as some of the other reconstructions, and according to D'Arrigo et al 2006, it's the only one for which the MWP was warmer than the portion of the 20th century it covered. See Table 2 from the paper below.

But what happens if you include the "current" temperatures that Wegman deleted? We don't have to wonder, because a graph showing these reconstructions with current temperature from the instrumental record is included right in the D'Arrigo paper. Here it is.

The top plot in the above graph contains the relevant information needed to evaluate whether recent decades have been warmer than the MWP. While it may well be that in Moberg's reconstruction the MWP had warmer temperatures than the portion of the 20th century it covered, by the end of the 20th century, temperatures were warmer than the MWP in any of these reconstructions. This is affirmed in Moberg et al 2005. In the abstract, they write, "According to our reconstruction, high temperatures—similar to those observed in the twentieth century before 1990—occurred around AD 1000 to 1100." In the body of the paper, they also say that "the post-1990 warmth seen in the instrumental data (green curve in Fig. 2b) appears to be unprecedented." Moberg et al 2005 directly contradicts what Wegman said about it. Now one has to wonder, why would Wegman adapt the time series in D'Arrigo et al 2006, delete current temperatures from them, and then make claims that two of these show a MWP warmer than current temperatures? And why did Wegman fail to report what D'Arrigo et al 2006 and these other papers actually said about these reconstructions? It would seem that Wegman was "hiding the incline" in current temperatures.

 

Conclusion

The Wegman report did not provide competent evaluation of the statistical methods in MBH98/99 or the critiques by M&M. Instead, after a restatement of M&M's claims that essentially ignored the literature showing them to be with out merit, the Wegman report decided to engage in largely ad hominem speculations about peer-reviewers without a shred of evidence actually coming from the peer-review process. Then he misrepresented the paleoclimate literature to make false claims on how warm paleoclimate reconstructions make the MWP relative to the end of the 20th century.

MBH98/99 were not perfect papers. Based on the evidence I've seen this would be my critical evaluation of the papers:

1. The supplementary material contained some clerical errors that added some difficulty to replicating MBH98. M&M found some of these and a corrigendum was published acknowledging this.
2. While biases associated with the MBH statistical method were small, they did exist. While they didn't significantly alter the shape of the hockey stick, they should be acknowledged.
3. The proxy data for the early centuries of the reconstruction was not as extensive as it could have been, so the confidence envelope was wider. While I think these papers made their case that the end of the 20th century was the warmest in the reconstruction, care needs to be taken to acknowledge the uncertainties in those early centuries. 

We now have far more extensive proxy networks and much more certainty regarding the last 2000 years, and the shape of the hockey stick hasn't changed much. Jim Java has produced a list on GitHub with citations, links and the most significant graph for dozens of proxy reconstructions that replicate the hockey stick on a hemispheric or global scale. At the time of this blogpost, there are 62 papers on this list. If there were a list of papers with global or hemispheric reconstructions that produced curves that weren't "hockey sticks," I'd be happy to share it, but these lists don't exist because the papers do not exist in the peer-reviewed literature.  Suffice it to say that, while much more work can and will be done to understand paleoclimate over the last 2000 years and longer, the "hockey stick" shape to the last 2000 years is no longer an area where serious scholars debate. There is no longer any rational objection to the fact that the 21st century so far has been the warmest in the last 100K years.

See my recent post about the Mann v Ball libel suit here. 

References:

[1] National Academies of Sciences, Engineering, and Medicine. 2006. Surface Temperature Reconstructions for the Last 2,000 Years. Washington, DC: The National Academies Press. https://doi.org/10.17226/11676.
https://nap.nationalacademies.org/read/11676/chapter/1

[2] Wegman, Edward J.; Said, Yasmin H.; Scott, David W. (2006), "Ad Hoc Committee Report On The 'Hockey Stick' Global Climate Reconstruction" (PDF), Congressional Report, United States House Committee on Energy and Commerce (published 14 July 2006).
https://climateaudit.files.wordpress.com/2007/11/07142006_wegman_report.pdf

[3] D'Arrigo, R., Wilson, R., and Jacoby, G. (2006), On the long-term context for late twentieth century warming, J. Geophys. Res., 111, D03103, doi:10.1029/2005JD006352.

https://agupubs.onlinelibrary.wiley.com/doi/epdf/10.1029/2005JD006352

[4] Esper, J. (2002). Low-Frequency Signals in Long Tree-Ring Chronologies for Reconstructing Past Temperature Variability. Science, 295(5563), 2250–2253. doi:10.1126/science.1066208
https://pubmed.ncbi.nlm.nih.gov/11910106/

[5] Moberg, A., Sonechkin, D., Holmgren, K. et al. Highly variable Northern Hemisphere temperatures reconstructed from low- and high-resolution proxy data. Nature 433, 613–617 (2005).

https://doi.org/10.1038/nature03265