On Misplaced Certainty and Misunderstood Uncertainty

I know the climate scientists know they’re right, but a little care is called for. It’s important not to play fast and loose with the figures – especially when criticising someone else for playing fast and loose with the figures!

In a post entitled Getting things right, Realclimate yesterday addressed a piece of rogue science conducted apparently in-house by an NGO. Gavin Schmidt wrote:

The erroneous claim in the study was that the temperature anomaly in 2020 would be 2.4ºC above pre-industrial. This is obviously very different from the IPCC projections… which show trends of about 0.2ºC/decade, and temperatures at 2020 of around 1-1.4ºC above pre-industrial.

But the chance of “temperatures at 2020” being 1.4ºC above pre-industrial seems to me pretty remote – certainly less than 2.5%, if Gavin is quoting within 2 sigma confidence limits, as is customary.

You’d think in a blog post titled “Getting things right” that it was pretty important to get things right…

So I posted a comment and was pleased to see not one, but two replies:

Now I’m confused. I understand we are currently about 0.8ºC above pre-industrial. A mean global surface temperature 1.4ºC above by 2020 implies a 0.6ºC rise over the next decade.

[Response: The range is just eyeballing the IPCC figure for the year 2020 – so there is some component of internal variability in there as well. – gavin]

[Response: GISS temperature of 2010 (which happens to be right on the long-term trend) is 0.9 ºC above the mean 1880-1920 (and the latter is probably a bit higher than “preindustrial”). -stefan]

OK, let’s take 0.9ºC, though that’s not a figure you often hear.

The IPCC graphic Gavin is referring to when he says “projections” is one I’ve never really liked:

It’s all a bit too imprecise and pretty for my liking. For example, the yellow line (constant GHG levels from 2000) diverges from the other scenarios almost immediately, even though natural variation would initially overwhelm differences between emission trajectories.

It does rather look, though, as if at least one of the scenarios could, according to the models, lead to warming of 1.4ºC above the pre-industrial level. Could this be because emissions in the scenario are much higher than we’re actually experiencing? No, Gavin notes that:

* Current CO2 is 390 ppm
* Growth in CO2 is around 2 ppm/yr, and so by 2020 there will be ~410 ppm
So far so good. The different IPCC scenarios give a range of 412-420 ppm.

The difference between 420ppm and 410ppm would only lead to a 0.1ºC extra rise in temperature over the very long term and even then the climate sensitivity (the eventual temperature increase for every doubling of the atmospheric CO2 level) would have to be on the high side – around 4ºC.

No, the problem is that the temperature hasn’t risen fast enough to 2010 for the more extreme modelling predictions in the IPCC figure for 2020 to be sufficiently likely any more. The IPCC graphic is out of date, plain and simple.

It’s a bit puzzling to be honest why Gavin used the IPCC graphic, because another Realclimate post today has trend-lines suggesting a much more accurate estimate of the likely global mean surface temperature at 2020 – around 0.2ºC higher than at present or around 1.1ºC above the pre-industrial level (as Stefan noted, 2010 is roughly on trend).

But how confident are we in this estimate? What is the range Gavin should have quoted?

Well, here’s the point: you can’t just express uncertainty by running a few models with slightly different starting conditions (the “Monte Carlo” approach) and discarding 2.5% at each extreme of the resulting distribution.

No, we have to actually think about what we’re doing.

It rather seems to me there are different kinds of uncertainty that we might want to consider when trying to predict the temperature “at 2020”.

What are the types of uncertainty we might need to take into account?

Parameter uncertainty
These are our “known unknowns”. In this case, we don’t actually know that the trend is 0.18 or 0.19ºC per decade as discussed at Realclimate. It looks like it is, but this could change when we get a bit more data – maybe we’ll find over a longer timescale that the real figure is 0.16 or 0.21ºC per decade. This makes us less certain about temperatures further out – at 2030 or 2050, say – than at 2020.

But a relatively short time into the future, parameter uncertainty is dominated by:

Calculable statistical uncertainty
Measurements of mean surface temperature show some variability about the underlying trend, as can be seen from the graphs in the Realclimate post discussing the data for 2010.

But the most any year has varied above the trend-line is about 0.2ºC in the case of 1998, which remains one of the 3 warmest years on record (with 2005 and 2010) due to the super El Nino that year. Maybe Gavin is implicitly including the possibility that there will be another strong El Nino in 2020. But that would only get us to a 1.3ºC total temperature increase (1.1ºC for the trend plus 0.2ºC for the El Nino), not 1.4ºC.

Statistical distribution uncertainty
It’s just conceivable Gavin calculated the Standard Deviation (SD) of annual temperature deviations from the trend and found it to be 0.15ºC or more so that 2 SDs includes 1.4ºC, so even if the long-term increase in temperature around 2020 is our 1.1ºC, there may still be a greater than 2.5% chance that the temperature in that one particular year is 1.4ºC or above. The only trouble is, with a mean of 1.1ºC and SD of 0.15ºC there would be an equal probability of 2020 being much colder than usual, so Gavin would have had to give a range of 0.8-1.4ºC.

Ah, but maybe Gavin expects the distribution to be skewed, so that freakishly hot years are more likely than freakishly cold ones…

The point is we don’t actually know a priori what the distribution of probabilities (often called the Probability Density – or sometimes Distribution – Function, or PDF, if that isn’t too confusing!) for the annual mean temperature of a given year actually looks like. We need a theory to tell us that – and the PDF could be complex, not a nice normal, lognormal or power curve at all.

Damn, we already have three sources of uncertainty compounding our estimate of the 2020 temperature!

It can’t get trickier than this can it?

Execution uncertainty
Yes it can.

Global temperatures are depressed following volcanic eruptions. It’s almost as if these are being ignored and that global warming projections include the implicit qualifier: “unless there’s a major volcanic eruption”. These are frequent enough for them to be included in our “2 sigma” (central 95%) range: volcanoes in 1963 (Mount Agung), 1982 (El Chichon) and 1991 (Pinatubo) depressed global temperatures by up to 0.3ºC. Despite a long-term warming trend, the temperature “at 2020” could easily be knocked back to 2010 levels, that is, 0.9ºC above pre-industrial, or below.

I don’t want anyone coming back and saying I predicted 2020 to be warmer than 2010 and it wasn’t. Sure, I could say “the theory was right, there was just that damn eruption”. But really we need to include the possibility of volcanic activity if we’re going to make a serious forecast.

I’m beginning to think 1-1.4ºC above pre-industrial might not be that good a prediction for 2020. It seems a volcanic eruption could push us further below our central forecast of 1.1ºC than a strong El Nino could lift us above. I suspect 2 sigma confidence limits are more like 0.8-1.3ºC, with the proviso that a really serious volcanic event could leave us even cooler, without the possibility of a corresponding extreme warming event.

The point, of course, is that uncertainty in a complex system, such as the climate or the economy isn’t likely to be a simple mathematical relationship. We need to explore the theory itself. We need qualitative as well as quantitative understanding.

Unknown unknowns
So far our 2020 temperature predictions have assumed we’re certain about our theory.

But maybe we’re not as smart as we think we are.

This is where it gets really difficult. Nevertheless, we should really have a look at any developments that are bubbling up. For example, Realclimate itself has discussed modelling that suggests there could be natural cycles that affect the temperature over timescales of decades. Personally, I think there could be something in this.

Again, the risks, according to the researchers, are to the temperature downside over the next decade. How sure are we that the groups looking at these patterns of variability are wrong? Not more than 95%, surely?

Let’s make one final allowance. Let’s take account of this unknown unknown and predict that the mean global temperature at 2020 will in fact be in the range 0.7-1.3ºC above the pre-industrial level, with a central prediction of a 1.1ºC rise. That is, it will be from 0.2ºC cooler than 2010 to 0.4ºC warmer, with a median expectation in the PDF of a 0.2ºC rise, so a skewed distribution. Think of the 0.2ºC drop as maybe some cyclical cooling cancelling out some of the warming trend plus a bit of volcanic action; the 0.4ºC warming would perhaps arise with a continuation of the current trend plus a big El Nino.

This is the point I want to make: the PDF is in large part a judgement, based on understanding (so there’s plenty of people who could make a better stab at it than me). Number-crunching on its own will never do the job.

I agree with the guys at Realclimate, though: it’s important to get things right!