On the Nature and Value of Consensus
About 20 years ago, I remember participating in a meeting where participants were actively debating a proposition. The moderator was enforcing parliamentary procedure, and speeches for and against the proposition were heard in turn - one speech for, then one speech against. This being the first meeting I had been in following this kind of procedure, I began to think that the room was evenly divided between the two positions. I was strongly in favor of it, and I was disheartened to see so much opposition to it. But when the vote was finally taken, I was more surprised by the results. The proposition passed by an extraordinarily large margin. In fact, it appeared that those speaking against the proposition made up almost all the people that voted against it, and the vast majority of the people in the room were like me. Most of us had essentially made up our minds, and we had no intention of making any speeches; we were just waiting for the debate to be over so we could vote.
My impression of the room before the vote was subject to a kind of selection bias. The proportion of people speaking for vs against the proposition was not representative of the positions held by those in the room. The rules of debate were set this way for at least two reasons: 1) to give all sides equal opportunity to argue for their position and 2) to help mitigate against people voting for the position that seemed most popular. You would have to make your decision based on weighing the arguments presented, not by simply considering which position was most popular. However, it would be fallacious to conclude anything about the proportion of the room in agreement with the proposition based on the proportion of people speaking in favor of it. The sample wasn't representative.
There is a similar dynamic affecting the climate debate in the public sphere. A subset of scientists are speaking out as proponents of or detractors to AGW. These scientists are showing up on TV news programs, giving congressional testimony, writing on blogs, and participating in various ways in the public conversation. They are doing so frequently at the behest of governmental or media entities that invite them to share. Others are just doing their work and publishing their research in the peer-reviewed literature, teaching courses in major universities, and collecting the data that can be used to assess AGW and its impacts on our future. The proportion of people speaking in these venues is inevitably going to be subject to selection bias - it's not a representative sample. In the public conversation, those with positions against AGW are inviting scientists who share their views to speak on their shows. Likewise, those with positions for AGW are doing the same. This selection bias can create a false impression that climate scientists are more evenly divided in their positions than they actually are, and this is one reason why the topic of "consensus" has become such a significant and divisive component of the public AGW debate.
Defining Consensus
There are some who object to the very notion of consensus. We hear claims like "there is no consensus in science." This, of course is preposterous. While consensus is almost never 100%, scientists can and do come to an overwhelming consensus on many points. In Geology, for instance, the theory of plate tectonics would be an example of this kind of consensus. While it took some time for the evidence to generate this consensus, there is now undeniably a consensus among Geologists on plate tectonics, and it has been revolutionary in the way scientists have understood many of the Earth's geologically significant processes.
However, in order to assess the nature and value of a consensus, we need first to definite it, and there are multiple ways to do so. But it's important first to decide what we're going to measure - what is it that we want to know if there is a consensus about? For the purposes of this post, I'm going to define this as, "human activity is the dominant cause for global warming above the 1850-1900 mean." I'm going to refer to this simply as AGW. In other posts, I've shown that the best evidence we have is that natural forcings have been more or less negligible over this time frame, and warming has been dominated by human emissions of GHGs, masked in part by aerosol pollution. This is the overwhelming position in the scientific literature, including IPCC assessment reports.
Second, we need a method to measure the consensus on this point. There are at least two ways we can do this. First, we can survey the opinions of scientists. You can find the results of surveys and petitions online, either of scientists expressing concern about AGW or of professionals (a few of which are scientists) saying there's no climate emergency. I don't find this approach persuasive, partly because it's hard to get a percentage of all qualified scientists having an opinion on something and partly because it focuses on the people, not the evidence. A second approach is to attempt to quantify the positions expressed in the peer-reviewed literature on AGW. This gives us a way to come up with a reliable percentage. Searching indexes of the peer reviewed literature, we can find the vast majority of papers taking a position on AGW, and we can then assess the percentage of the literature that agrees with AGW. This has the benefit of focusing on the literature where primary research is published, so it gives us a sense of what the published scientific evidence is currently saying. It has the weakness of being time an labor intensive to quantify. Teams of people have to read papers (or at least abstracts) to come up with the percentage consensus on AGW. This means any result is going to be an estimate with an error - there is the possibility of a paper being wrongly categorized, either as to whether it expresses a position on AGW or whether it agrees or disagrees with AGW. So any number we arrive at will be somewhat approximate. But this approach does the best at focusing on the published evidence concerning AGW rather than on the opinions of people.
Existence of Consensus
There are two papers recently published that attempt to quantify and assess the percent consensus about AGW following the second method. This method focuses more on the published evidence and less on the people. The first received a great deal of attention about 10 years ago and has been repeated (sometimes incorrectly) in political discussions. The second is is a follow up to the first, and in my opinion it is a better paper.
Cook 2013
John Cook[1] undertook the task of attempting to quantify the amount of published research that agrees with AGW. The study consisted of two parts. In part 1, Cook and his team collected and evaluated 11,944 abstracts to papers that mentioned "global climate change" or "global warming" published between 1991 and 2011. Most of these papers didn't address whether human activity was the dominant cause of global warming. Only about 33.6% took a position at all - "32.6% endorsed AGW, 0.7% rejected AGW and 0.3% were uncertain about the cause of global warming." Most of the papers trapped by their search were irrelevant - they didn't take a position on AGW. Of those papers that took a position on AGW, 97.1% endorsed AGW.Two contrarian methodologies above want the "consensus" to be defined either as the sum of explicit and implicit endorsement columns or only the value in the explicit endorsement column. If we were to take these definitions of consensus and apply them to plate tectonics, the consensus would be either 29% or 0%, and the consensus on AGW would be 32.6% or 0.5%. But the actual consensus values are ~100% on plate tectonics and ~97% on AGW.
Lynas 2021
Value of Consensus
One of the biggest concerns I have about this whole enterprise is, why do we have to quantify the level of consensus there is on AGW? There are very few scientific issues where consensus is quantified. The closest equivalent I can think of is "Project Steve." In response to lists of scientists who doubt evolution, Project Steve responded with a tongue in cheek list of scientists named Steve (about 1% of scientists) who accept evolution.[5] But in most issues, there is no need to quantify any consensus. Once the evidence reaches a critical mass, scientists come to see matters as "settled," and move on to other matters. Those matters that had been "settled" can certainly be reconsidered if new, contradictory evidence comes in, but rarely if ever is there any need to quantify a consensus or how settled a scientific issue is.
So why do we do this with AGW? I believe the strongest answer to this is that if AGW is correct and dangerous, it's critical that we act quickly to keep the most dangerous effects of AGW from happening. Since some are saying (wrongly) that AGW is still a matter of open debate and uncertainties are still large, Cook and others have undertaken the task of showing that this in fact is not the case. The scientific research has achieved a near unanimous conclusion on AGW. The consensus is being quantified for political reasons so that voters and policy makers can see that the impression they may be getting in media presentations and congressional testimony is subject to a strong sample bias. The debates we see on news shows and in government testimony are not representative of the whole of scientific research. And while it may be valid to hear the best arguments of both sides, the reality is that one side is vanishingly small and shrinking, and the evidence no longer supports continued scientific debate on AGW. The clear weight of the peer-reviewed evidence is nearly unanimous in favor of the consensus. However, would like to clarify what this does and doesn't mean.
- The greatest value of these consensus papers is that it makes clear that AGW is an issue about which there is no longer any meaningful debate. Climate scientists, for the most part, are not debating whether human activity is the dominant cause of warming above the 1850-1900 mean. They have moved on to other aspects of climate science.
- The fact that consensus exists and we can consider AGW "settled science" is not proof that it's correct. Further evidence could in principle come in that would overturn the current consensus.
- The fact that consensus exists on AGW doesn't mean that there no aspects of climate science that are less certain. Scientists are still working to constrain estimates for ECS, understand how quickly ice sheets may collapse, how AGW will affect tropical storms, etc. Nobody is suggesting that all matters related to AGW are settled or that there are no remaining areas of uncertainty.
- Aside from the political motivations behind Cook 2013, there was never any reason to quantify the consensus, as best I can tell. In that sense, Richard Tol is right. The fact that a near unanimous consensus exists was already obvious before any of these studies were conducted, and we didn't need this study to know that.
- However, Tol was wrong to say that no one ever said the consensus was not in the high nineties. People say that frequently and still do, and the fact that they do is part of the reason why the consensus was quantified. The publication of a quantified consensus may well have increased the volume of those who say there is no consensus on AGW. Cook's paper was a response to a clear denial that a strong consensus exists already present among the public and politicians.
- In debating AGW, it's always wrong to use these stats for or against the validity of AGW. AGW is almost certainly correct because the evidence overwhelmingly supports it, not because the consensus exists.
- Focusing on the percent consensus can breed conspiratorial thinking and polarize the public. Some people get the impression that there are forces looking to enforce a consensus and people fall in line with it, rather than simply quantifying a consensus that exists because of the evidence.
- We should not expect policy makers to be scientifically literate or keep up with the literature, and as much as I dislike the notion of quantifying the consensus, I can see it as being valuable for those willing to listen to the evidence. However, it seems to me generally speaking that the 97% statistic has only been persuasive for those that were already persuaded. Those who reject the consensus for political reasons are not likely to accept it because they're told they reject what almost all scientific research supports.
- Consensus may well exist on matters of climate science that were not quantified in the above studies, particularly in areas of the expected consequences of AGW.
- The best consensus documents are still the IPCC assessment reports, which do more than just survey the literature for statements affirming AGW. They quantify the certainties with respect to the evidence we currently have regarding various aspects of climate change, including its attribution to human activity, ECS and the various dangers associated with AGW.
References:
[1] John Cook et al, 2013, Environmental Research Letters 8 (024024). DOI 10.1088/1748-9326/8/2/024024
Comments
Post a Comment