Error propagation in climate models

An important new article, by chemist Patrick Frank, was published on Friday in Frontiers in Earth Science. In essence, it demonstrates that none of the climate prediction models currently in use is capable of making any predictions whatsoever about anthropogenic CO2 warming, because their cumulative error-bars outweigh what they seek to predict by an order of magnitude. They are therefore used illegitimately to predict climate change. This would seem to be serious problem.

Frank is a senior researcher at Stanford, with 54 publications to his credit. He specialises in the validation of predictive models in science, particularly in chemistry. And yet it has take 6 years to get publication, because a majority of climatology journals simply did not respond to his submission, and of those that did, publication was recommended by peer reviewers outside the climate prediction field, but the paper was castigated by those within the field, usually a majority. He points out that he has seen nothing else like this in over thirty years of scientific publication. The rejections, then, appear to have been about the damning conclusions, not the science.

The paper, it is important to note, is not about climatology itself, but about the long-overdue testing of the models, in particular by a standard scientific technique called error propagation. Frank points out, with regret, that the majority of climatologist reviewers demonstrated undergraduate-level misunderstanding of the difference between the “precision” and “accuracy” of a model. As the paper’s Introduction states:

In his evaluation of climate predictions Smith noted that, “[E]ven in high school physics, we learn that an answer without “error bars” is no answer at all” (Smith, 2002). However, projections of future air temperatures are invariably published without including any physically valid error bars to represent uncertainty. Instead, the standard uncertainties derive from variability about a model mean, which is only a measure of precision. Precision alone does not indicate accuracy, nor is it a measure of physical or predictive reliability.

In other words, those pretty graphs in IPCC publications, showing a range of possible outcomes, are only showing the degree of (im)precision when the particular model is run repeatedly, assuming the truth of the model itself. But tests for the uncertainty within the model itself may demonstrate (and do) that, however precise it is in the former sense, it may have no predictive ability whatsoever – it is unreliable as a tool.

Such tests, Frank says, should have been done thirty years ago, when the field, and the hysteria, first began, but they never have been, or things would never have become as fraught as they are today.

To cut to the chase, climate models admit to a high degree of uncertainty about the important factor of cloud cover from year to year. Even if they did not admit to it, the large errors in past predictions of cloud cover, which can be shown by comparing all the models against the data, demonstrate the fact. But since the prediction for cloud cover becomes part of the model’s input for its next round of prediction, the level of uncertainty becomes rapidly bigger until, very quickly, it greatly outweighs the small changes being predicted. The net result is clearly shown in this graphic from the paper:

The left side shows the familiar output of models using past data (the solid lines show how the models themselves may be modelled by a simple linear function – explained in the paper’s text): the right side shows the same at the scale necessitated by adding the error bars. Whereas the difference between world temperature based on different CO2 concentrations was maybe 2 degrees from 1960-2020, the uncertainty bars, at +/- 15 degrees, make those differences meaningless: the model can predict nothing it purports to predict even a year or two ahead, let alone half a century.

Frank’s critics, it seems, mainly failed even to appreciate that these large errors do not represent actual temperatures (ie, that the world may warm or cool by such large figures on the models) but rather the gross uncertainty of the predictions they do make, since the same models predict the impossibly hot, the impossibly cold, and everything in between.

My own disillusion about climate models came with the realisation that their predictions have never been validated by data: instead, data have frequently been doctored retrospectively to fit the models, and the models’ mutual consistency (actually dictated by their assumptions) has been substituted for proper validation. The statistical criminality of this kind of abuse of modelling was one of the themes of William Briggs’s book Uncertainty, so I am maybe particularly sensitive to it despite my poor mathematical grasp of statistics. It is all part of the big picture of uncertainty (aka chance) in science which has interested me for several years.

To a mathematically challenged medic like me, the detail of Frank’s paper is hard to grasp, though the general points, and the evidence used, are clear enough. Fortunately, Frank speaks in a YouTube video on a previous version of the paper, back in 2016. The main points are very clear in this, and those with sufficient background can adjust to any alterations in the final paper or its downloadable information base.

Both deserve wider notice than they will probably get in the current climate (!). The fact that he ends the video by acknowledging, wryly, that didn’t receive oil company money for this work, but paid for it himself, shows the level of ad hominem in this murky field of politicised science. The author expects that there will be no valid refutation of his work, but much trashing of his character, in the wake of publication.

Patrick Frank is a scientist, not an activist: it is the objectivity of science that appeals to him. Yet his own treatment in getting his work published, and what it has all meant in terms of public anxiety and harmful policies, is clear from the blog post he published yesterday, announcing the paper and explaining the background. His anger comes through clearly, and I can’t disagree, for the failure envelops the whole of science, government and our whole culture:

[A]ll the frenzy about CO₂ and climate was for nothing.

All the anguished adults; all the despairing young people; all the grammar school children frightened to tears and recriminations by lessons about coming doom, and death, and destruction; all the social strife and dislocation. All the blaming, all the character assassinations, all the damaged careers, all the excess winter fuel-poverty deaths, all the men, women, and children continuing to live with indoor smoke, all the enormous sums diverted, all the blighted landscapes, all the chopped and burned birds and the disrupted bats, all the huge monies transferred from the middle class to rich subsidy-farmers.

All for nothing.

There’s plenty of blame to go around, but the betrayal of science garners the most. Those offenses would not have happened had not every single scientific society neglected its duty to diligence.

From the American Physical Society right through to the American Meteorological Association, they all abandoned their professional integrity, and with it their responsibility to defend and practice hard-minded science. Willful neglect? Who knows. Betrayal of science? Absolutely for sure.

Had the American Physical Society been as critical of claims about CO₂ and climate as they were of claims about palladium, deuterium, and cold fusion, none of this would have happened. But they were not.

The institutional betrayal could not be worse; worse than Lysenkoism because there was no Stalin to hold a gun to their heads. They all volunteered.

These outrages: the deaths, the injuries, the anguish, the strife, the malused resources, the ecological offenses, were in their hands to prevent and so are on their heads for account.

In my opinion, the management of every single US scientific society should resign in disgrace. Every single one of them. Starting with Marcia McNutt at the National Academy.

The IPCC should be defunded and shuttered forever.

And the EPA? Who exactly is it that should have rigorously engaged, but did not? In light of apparently studied incompetence at the center, shouldn’t all authority be returned to the states, where it belongs?

And, in a smaller but nevertheless real tragedy, who’s going to tell the so cynically abused Greta? My imagination shies away from that picture.

But as cynical folk note, nobody will tell Greta – she will still tour the world telling people to listen to the scientists, but meaning listen to the activists.

Avatar photo

About Jon Garvey

Training in medicine (which was my career), social psychology and theology. Interests in most things, but especially the science-faith interface. The rest of my time, though, is spent writing, playing and recording music.
This entry was posted in Politics and sociology, Science. Bookmark the permalink.

6 Responses to Error propagation in climate models

  1. Avatar photo GD says:

    Hi Jon,

    I gave a presentation to a group at the University at about 2001 where I considered atmospheric chemistry and major cycles (carbon), to show how climate models are error prone. This paper goes to the numbers in the models, but overall is consistent with my analysis. I have not been subjected to the treatment indicated (probably because the chemistry is so complicated that no-one could get into a discussion).

    I will add however, that the increases in atmospheric GHG is real, and qualitatively measures are needed to prevent a runaway increase.

    • Avatar photo Jon Garvey says:

      Interesting agreement, George.

      The paper is, of course, about methodological failure (and points to the likelihood of theoretical error without attempting to specify it).

      The question of runaway greenhouse effect is, as you say, another matter. But it still has to cross the theoretical hurdle of unknown feedback, and the pre-historic indications that carbon dioxide levels have been very much higher in the past, without such a runaway effect (and also the indications from the past that CO2 has lagged temperature change, rather than leading it).

      Be that as it may, it would be nice to use some of the plentiful coal without emitting lots of GHGs – over to you for that!

  2. Avatar photo GD says:

    …. the pre-historic indications that carbon dioxide levels have been very much higher in the past, without such a runaway effect (and also the indications from the past that CO2 has lagged temperature change…

    Agreed (changes were to the planet), but a big difference is the human population that is ever growing and using up more of the environment. I view overall a slow motivation to change on a planetary scale to an environmental, and human habitation, improvement …. but with many bumps on the way.

  3. billygodzilly says:

    Hi, Jon.

    Very nice coverage and summary of Pat’s recent publication.

    I wondered if you knew that Pat had posted a synopsis (a rather detailed synopsis) on Anthony Watts website WattsUpWithThat. It has generated quite a lot of discussion.

    I thought that you might like to read it.


  4. Avatar photo Jon Garvey says:

    Welcome to The Hump, billygodzilly.

    I did in fact find my way to the paper via the Watts site, but like when possible to get folks to look at original articles first, if they have the background. I would certainly suggest readers here who are interested look at the generally well-informed and constructive discussion there.

Leave a Reply