Debaters Behaving Badly, Part 4 - Inaccurately Comparing Datasets

In previous posts in this series, I've tried summarize what I consider bad behavior among those debating climate change. So far I've discussing what I consider statistically unethical practices - using short-term trends, misusing scales, and using local instead of global data. Here I'd like to look how we make comparisons between types of data.

In climate science, we need to deal with all sorts of different kinds of data. For global temperatures, I can think of at least 3 - proxies, the instrumental record (satellites, thermometers), and models (reanalysis, modeled projections, etc). It's perfectly fine to create graphs that include all of these kinds of data, but in doing so, we have to make sure that we are communicating honestly and making comparisons accurately. Proxy data has lower resolution and larger confidence intervals than the instrumental record, and modeled projections always depend in part on the assumed scenario, and confidence intervals for these projections grow with time. My interest in climate change is mostly rooted in my love of geology, so for me I love looking at proxy evidence and how current temperatures compare to the past. But I'm not a fan of seeing graphs like this.


Marcott 2013 with the Instrumental Record and Model Projections

It's not so much the humor that bothers me (calling the preindustrial Holocene a "bunny slope" is funny). But there are several things here that I do take issue with:

  1. The GMST y-axis is set to a 1961-1990 baseline, but it's generally accepted to use an 1850-1900 baseline for modeled projections so that we can be consistent with how the values we report compared to this proxy for preindustrial temperatures. Not a big deal; just a pet peeve.
  2. The data for Marcott 2013 ends in 1940 (20-year centered mean), so it's unclear to me how they set that to the 1961-1990 mean. It for certain can be done, but to do so you'd have to first find a common baseline between Marcott 2013 and the instrumental record and then shift both by an equal amount.
  3. The name of the instrumental dataset is not given nor whether smoothing is applied.
  4. The confidence envelope for Marcott 2013 is given, but not the confidence envelope for the modelled projections.

The last one is the most egregious to me. If you want to help me understand how much warming above the warmest temperatures of the preindustrial Holocene we can expect by 2100, I need to know know the CI for the model. Without providing it, it looks like you're trying to make the graph look as scary as possible. Here's another example.

This graph gives the impression that Marcott 2013 contains recent temperatures. It's labeled as if that spike in temperatures is part of Marcott's proxy reconstruction. It's not. His graph ends in 1940, and even the 5'x5' reconstruction does not show this much warming. In my view best practice with his study is to use the RegEM reconstruction, which using infilling and matches the instrumental record more closely. I have a summary of the issues involved with Marcott 2013 here and here, including how robust the last 50 years is and whether possible large spikes in global temperatures would be smoothed out by Marcott's statistical methods. But the author of this graph then plotted 2014 and 2015 from HadCRUT4. However, Marcott 2013 has a resolution of 20 years, so plotting individual years on this graph can give false impressions about how much warmer we are now than the HTM (in this graph, about 6.5K years ago). We can do better than this.

Marcott 2013: A Better Approach

I think if we're going to make these kinds of graphs, we need to follow some pretty basic and standard guidelines to ensure that are presenting the data as honestly as possible. Here are some guidelines I follow:
  1. Plot confidence intervals if available, especially for proxy and modeled data.
  2. Label the types of datasets used. It should be clear what is proxy, what is instrumental, and what is modeled.
  3. Match smoothing across datasets. Marcott's data has a resolution of 20 years, so plots of the instrumental data need to match that resolution.
  4. Set the scales of the x and y axes to allow for the most clarity of presentation, and be clear about the baseline used.
Here is an example of graphs I made using Marcott 2013 and HadCRUT5 data. Notice here that the proxy data has the confidence envelope shown as dotted lines and HadCRUT5 is shown in black, so it's clearly distinguishable from the proxy data. I labeled the graph with the smoothing and baseline clearly indicated.
Before publishing this on my blog, I also checked to see if what I was plotting made sense. Here is the overlap between Marcott 2013 proxy and the HadCRUT5 data.
The above graph shows HadCRUT5 both with 20-year smoothing and without. The overlap with Marcott 2013 is the dots (my spreadsheet didn't automatically connect them). As you can see, though, even the annual data fits in the confidence envelope for Marcott 2013 using the RegEM reconstruction. Even though the overlap isn't for that many years, it does give me confidence that I've matched both to the 1850-1900 baseline accurately and the two agree with each other when they overlap. So using my first graph above with Marcott 2013 and a 20-year smoothed HadCRUT5, we can see that current temperatures are ~1 C above the 1850-1900 mean and about 0.3 C warmer than the upper confidence envelope for the HTM. So I feel confident in saying that current temperatures are "likely 0.3 C warmer than at any point in the preindustrial Holocene." Warming is continuing at 0.2 C/decade, so we are leaving the temperatures of the preindustrial Holocene in the dust at a rapid pace.

Pages 2K: A Better Approach

We can do the same thing with the Pages 2K dataset and HadCRUT5. I plotted both to the 1850-1900 mean. Since Pages 2K has annual resolution, I plotted HadCRUT5's annual data. Using the confidence envelope for Pages 2K, I get the following results.
Here I zoomed in on the overlap between the two datasets. You can see remarkable agreement between them, but HadCRUT5 definitely shows more variability than Pages 2K during the years they overlap. However, none of the HadCRUT5 temperatures fall outside the confidence envelope for Pages 2K.

From this, we can see that current temperatures are about 1.2 C warmer than the 1850-1900 mean. The warmest preindustrial temperature in Pages 2K (the top of the confidence envelope for the warmest years) is about 0.8 C. Because of this, I can feel confident that I'm being accurate to the data by saying that currently GMST is likely 0.4 C warmer than at any point in the last 2000 years, with temperatures currently increasing at 0.2 C/decade.

Conclusion

As a general rule of thumb, I think it's advantageous to make claims that can't rationally be accused of "alarmism." I'm not going to post model projections without confidence intervals. I'd much rather stick to the empirical data and state what can be likely concluded from that data accurately. I'll still be called an alarmist, but those that do will have no rational basis for saying so.









Comments

Post a Comment

Popular posts from this blog

The Marketing of Alt-Data at Temperature.Global

Image
Temperature.Global (12-month running average, sort of) A few years ago, I started seeing people post links to a new dataset for global temperatures at a website called temperature.global. There are at least four unique features of this dataset. First, the website claims to update global temperature in near real time. Second, every year in the dataset averages below "normal." Third, there is absolutely no transparency regarding their methodology (those who administer it are anonymous, most of the data is hidden, and much of their methodology is unclear). And fourth, what is disclosed about their methodology reveals that what they are calculating is not the Earth's surface temperature. Description of Temperature.Global If you go to the temperature.global website, it becomes almost immediately apparent that this website has little concern for transparency. The website is hosted by Parler (the alternative social network), which if nothing else is pretty odd. There are no name...
Read more

Roy Spencer on Models and Observations

Image
A few days ago, Dr. Roy Spencer wrote a piece for the Heritage Foundation called, " Global Warming: Observations vs. Climate Models " ( PDF ) essentially arguing that models show too much warming compared to observations, and if we stick to observations, "global warming offers no justification for carbon-based regulation." He claims to frame his argument in terms of answering three questions: Is recent warming of the climate system materially attributable to anthropogenic greenhouse gas emissions, as is usually claimed? Is the rate of observed warming close to what computer climate models—used to guide public policy—show? Has the observed rate of warming been sufficient to justify alarm and extensive regulation of CO2 emissions? We should keep in mind that this is a political document intended to support the political aims of the Heritage Foundation, and Spencer has carefully selected what he says and doesn't say to fit the political agenda of the Heritage Found...
Read more

Patrick Frank Publishes on Errors Again

Image
Recently I came across yet another paper by Patrick Frank[1] attempting to claim that climate scientists have been underestimating uncertainties in climate-related data. In this paper, he takes aim at GMST data, and he argues that  LiG resolution limits, non-linearity, and sensor field calibrations yield GSATA mean ±2σ RMS uncertainties of, 1900–1945, ±1.7 °C; 1946–1980, ±2.1 °C; 1981–2004, ±2.0 °C; and 2005–2010, ±1.6 °C. Finally, the 20th century (1900–1999) GSATA, 0.74 ± 1.94 °C, does not convey any information about rate or magnitude of temperature change. The resulting GMST graph from his calculations is below. Essentially, he's saying that errors associated with liquid in glass thermometers are so large that we can have no confidence in the global warming trend in the major GMST datasets. Of course, the organizations producing these GMST datasets all evaluate the uncertainties associated with their anomaly values, and their estimates are invariably much smaller - about ±0.05°...
Read more