Does NOAA have "Ghost Stations" for US Temperatures?

The Epoch Times (ET) is a newspaper operated by the Falun Gong cult which was formed in 1992. The cult is (understandably) opposed to the Chinese Communist Party, but recently it has made some inroads into the US, where it supports far-right political agendas; its ET newspaper has promoted a number of conspiracy theories involving QAnon, anti-vaccine propaganda, and climate science denial. In a recent "premium report," the Epoch Times claims that the U.S. Temperature dataset operated by NOAA contains data from "non existent temperature stations" and asserts that there are "hundreds of ‘ghost’ climate stations" that are no longer operational, and data is just filled in from surrounding stations, with the implication that they do so to support Biden's climate policies.

The Supposed Problem

The article relies mostly claims made by influential climate-tweeter named John Shewchuk and climate-blogger Anthony Watts. Strangely, almost all their claims have to do with the now-deprecated USHCN dataset, and as we'll see, they had to use now defunct datasets in order to make their lies seem plausible. The USHCN v2.5 dataset was built on a 2.5° longitude by 3.5° latitude gridded analysis of monthly temperatures from the 1,218 stations in the USHCN network. This was the standard NOAA dataset for CONUS temperatures until about 2014. 

In 2005, NOAA introduced the USCRN network, which is made up of 114 ideally-sited stations for monitoring changes in CONUS temperatures. From 2005 to 2014, NOAA monitored both USHCN and USCRN and documented a remarkable agreement between these two datasets over that decade. And in 2015, the USHCN dataset was replaced by a new nClimDiv dataset, which is built on a finer grid (about 5 km by 5 km) and is derived from the global GHCN-daily network. This network contains "several thousand more stations" (nClimDiv has over 10,000 stations) that are available in GHCN-daily. In addition, nClimDiv uses stations in Canada and and Mexico near US borders for the interpolation of US anomalies in areas near the US border. So in 2014, NOAA officially retired USHCN, and it's no longer maintained, meaning that any plots of USHCN station data following 2014 will not be representative of anything. Instead, NOAA uses USCRN (a low resolution dataset with 114 ideally-sited stations) and nClimDiv (a high resolution, homogenized dataset with over 10,000 stations) to monitor changes in CONUS temperatures. Both USCRN and nClimDiv generally agree with each other, though USCRN shows marginally more warming than nClimDiv during the time where they overlap (2005 to present). 

USCRN shows Marginally More Warming than nClimDiv

Over the years, Tony Heller has used both raw and bias-corrected station data from the USHCN to assert that NOAA has been engaging in criminal activity.  He's developed several freely available tools that others can use, but they are all based on the old USHCN v2.5 dataset. I don't know why he hasn't updated himself to the newer nClimDiv dataset; he could have at any point in the last decade. Perhaps the data requirements of using 8 times the stations were too great for him, or perhaps he has tried to upgrade and found that his claims don't hold up as well in the new dataset. But he's a decade out of date, and anyone using his tools will be out of date as well. If Shewchuk (and others at CO2 Coalition) are dependent on Heller's tools, this may partly explain why Shewchuk and other contrarians continue to use the older dataset even after it's been replaced and is no longer maintained.

Yet with this understanding of the two current datasets, it can be easily seen why the claims by Shewchuk reported in the ET are clearly false. In fact, it would appear that Shewchuk is deliberately continuing to use USHCN in order be deceitful. But let's make sure we get his accusations (as reported by ET) correct. According to Shewshuk, NOAA increased the number of stations until it reached 1218 stations in 1957, but then in 1990, the number of stations began to decrease as equipment failed and people retired. But as those stations dropped from the network, NOAA still took temperature readings for those stations, inferring the temperature for that station from surrounding station data. According to him, these stations “are physically gone—but still report data—like magic.” He refers to these stations as "ghost" stations (I believe Heller calls them "zombie" stations). Shewchuk is essentially saying that  NOAA is closing USHCN stations and then fabricating data for those stations, when overall NOAA has increased the number of stations so that they can produce higher resolution CONUS temperature data with nClimDiv.

Evaluation of this Supposed Problem

Is there any truth to what Shewchuk is saying about USHCN? Well, mostly "no." Yes, stations have dropped from the USHCN network, but NOAA isn't "fabricating temperature data" with "ghost" or "zombie" stations. What NOAA is actually doing is called "infilling," and it's statistically identical to calculating area-weighted averages. And infilling does not have a significant effect on CONUS temperature trends. To illustrate what's happening, let's set up a hypothetical grid with one station at the center of each cell in the network; this grid has 117 cells with 117 stations (similar to the USCRN network with 114 stations). We'll call this USCRN-like grid a "large cell grid" with the temperature of any cell in that grid the X temperature.

Hypothetical USCRN-like Grid with 117 X Stations

If you record temperatures at each X in the grid, then you can easily calculate the average temperature of the entire grid by averaging the values for each station. If your grid has cells of different sizes, you can multiply each cell temperature by the area of the cell and calculate an area-weighted average for the entire grid. But let's say we want a grid with finer resolution. Let's break up each of the 117 cells above in to 9 cells. If we do that, we can then "infill" these smaller cells with the temperature recorded in the center cell. If we do that, we'll get the exact same answer for the average temperature of the grid. For instance, see below. The average temperature of this grid is 20.50 C.

Simple "Infilling" of X-station Data to a Higher Resolution USCRN-like Grid

We haven't changed the average temperature at all by infilling the X temperature into these smaller cells. However, we can see weaknesses in this approach. For instance, if the X temperature for one cell is 15 C and the X temperature for an adjacent cell is 24 C, it's not likely that the transition from 15 C to 24 C is abrupt; there's more likely a more gradual change, and in a finer grid there's likely to be an incremental change between 15 C and 24 C. How can we handle that? Well in our hypothetical network we also have other weather stations that can help us fill out the temperature of the whole grid with finer detail. But if we put these stations onto this finer grid, not every cell is filled. So let's construct a "small cell grid" in which 9 small cells corresponds to the 1 large cell above. This will be analogous to a USHCN-like grid. It contains X temperatures (same as above) and O temperatures from stations not in the 117 in my previous example. Essentially, we've added 468 USHCN-like "O stations" for a total of 585 stations, leaving 468 small cells with no stations.

Hypothetical USHCN-like Grid with 585 stations and 468 empty cells

If we use all the USCRN-like X stations and the USHCN-like O stations in a small cell grid, we can get better resolution, but since we've adopted a finer grid, we now have "empty" cells. The process of "infilling" simply fills in those empty cells with data extracted from the surrounding cells. In this hypothetical USHCN-like small cell grid, I used the same X station data as above and added hypothetical O station values. Then I infilled the empty grid cells by simply averaging the temperatures recorded at stations surrounding the empty small cell. And while this is clearly an oversimplification of infilling within the USHCN network, I think it illustrates the rationale behind it quite well. In my hypothetical example, the average temperature of this small cell grid is 20.51 C, essentially unchanged from the large cell grid above. And infilling does not generate new data. The sample size N is not increased by infilling. In the USHCN, infilled values are clearly marked as such to indicate that these values are estimated, not measured and reported by stations.

Infilling empty cells in USHCN-like Grid from Surrounding Stations

Now what happens if a small cell loses its station for a period of time? Well, the best way to handle that is to infer the temperature of that small cell from the surrounding cells. In effect, all that's happening is that the area weight of the surrounding cells expand to fill in that cell when data from that cell is missing. And notice in reality here, given that there are more, smaller grid cells in the USHCN-like small cell grid, there's actually a reduction of infilling compared to the USCRN-like large cell grid, if you were to break up large cell grid with the resolution of the small cell grid. If we impose a small cell grid on the large cell grid above, then infill based on the 117 stations in the USCRN-like large cell grid, then 936 cells are infilled, or about 89% of the small cells. If we use the USHCN-like small cell grid with all 585 stations, then only 468 cells are infilled or about 44% of the small cells. This is clearly an improvement over the large cell grid with only 117 stations. Because you're inferring temperatures from more stations instead of from fewer stations, overall you're getting a better understanding of the geographic distribution of temperature data without a significant change in the overall trends in temperatures. However, infilled values are not data points. The sample size N is the number of reporting stations; infilling only serves to provide a better area-weighted average.

Zeke Hausfather's Work

So far this has been entirely hypothetical. I've explained the concept and rationale behind infilling. I've also shown why this isn't evidence of any sinister plot on the part of NOAA, nor does it imply NOAA is using "ghost" stations to manipulate temperature data. But can we show that my hypothetical example above accurately represents what is going on with the (now-deprecated) USHCN stations? It turns out that this has been done already by Zeke Hausfather about 10 years ago. Hausfather has shared this in several places, but I'll share it from a post he contributed to Judith Curry's blog so you can see that this is not something that even informed contrarians really take issue with. Here Hausfather calculates how CONUS temperatures with and without the infilled data. The difference between the two is plotted below.

After 1915, the Effect of Infilling is Negligible

ln the above graph, after about 1915, the trend in infilling adjustments is essentially flat, meaning that this has virtually no effect on CONUS temperature trends. As Haufather explains, infilling essentially just adds the climatology of the station with missing data to the area-weighted anomaly of nearby stations, making the effect essentially the same as area-weighted averaging. It's essentially saying area weight for surrounding stations increases to "fill in" the area during times when the station doesn't report data.  Whether you use a network with 114 stations (USCRN) or with 10,000 stations (nClimDiv) you get very similar results for US temperatures, so so we shouldn't expect that losing a few stations out of a network with 1200 stations (USHCN) and then using infilling would change US temperature trends anyway. Infilling quite simply has no significant effect on CONUS temperature trends, even within the now-deprecated USHCN network.

Conclusion

And of course, the USHCN network is no longer maintained by NOAA. In it's place is the nClimDiv dataset with over eight times more stations and a grid with much greater resolution, so NOAA has been increasing, not decreasing, the number of stations overall. ET knows about this update, but it quotes Anthony Watts essentially lying about the newer dataset. Watts is quoted as saying, "The USHCN data set and the [new] nClimDiv climate division data set [which uses the same stations and has the same problems]." I suppose it's superficially true that nClimDiv uses the same stations current in USHCN, but Watts neglects to mention that nClimDiv also adds thousands more stations not used by USHCN. And while nClimDiv uses the same algorithms and corrections applied to USHCN v2.5, neither Watts nor Shewchuk have demonstrated that there are any "problems" with these algorithms and corrections that would cause nClimDiv to report any spurious trends. In fact, the rural-only and ideally-sited USCRN network shows marginally more warming than nClimDiv when they overlap. Certainly we can say that the bias correction and infilling techniques used by nClimDiv are not causing any spurious warming with respect to USCRN. There is simply no evidence of any "ghost" or "zombie" stations that are being manipulated by NOAA to affect Biden's climate policies, or any policy whatsoever.