Judith Curry: trying your best is not the same as delivering something fit for purpose
Judith Curry had a post today which just made my blood boil when she tried to excuse the appalling culture within those groups producing a “global temperature figure” by saying “They are doing their best”. In my experience, that comment is usually only said when someone has done an appalling bad job – with poor quality materials, methods and training. So I posted a quick list of what I thought was needed:
1. Fully audited methodology and systems
2. Quality assurance to ISO9000
3. Some come back WHEN we find out they weren’t doing the job to the standard required that doesn’t involve putting them in jail.
4. Accountability to the public – that is to say – they stop saying “we are doing our best” and start saying “what is it you need us to do”.
Before I go much further I would be interested in some feedback. At the heart of my proposal is the need to remove the compilation of a “global temperature” figure from the current academics who just don’t seem to be up to the job. So what would we have instead?Impartiality as part of its ethos.
That the production of global temperature figures is carried out in an organisation which is rigorously impartial in that it does not in any way either privately or publicly suggest that rising, falling or any other kind of trend or indeed no trend is good, bad or indifferent. And that such a requirement is written into the contract of each and every employee.
Self-Auditing as part of its ethos
That as part of its normal reporting, it publishes details of any forecasts or projections together with a current assessment of those forecasts or projections against actual measured data.
Public scrutiny as part of its ethos
That as part of its everyday operations, that it makes available all aspects of its work to public scrutiny. That it enables members of the public to replicate its own figures where ever possible.
Quality Assurance as part of its ethos
That the organisation should have the relevant assessment by ISO9000 of its own quality assurance system.
That funding is tailored to match the organisations aims and not the organisation aims tailored to match funding.
In the past we saw examples of climate academics producing the global temperature on a spread sheet from third party. As such it was obvious that little or no attempt had been made to audit the suitability of temperature stations for the intended job. As the surface station project found, these were simply not up to the job.
As such it was obvious that funding was totally inadequate for the task at hand, and there was a culture of “make do and mend” for temperature data, rather than “get the data right first time” as it the requirement.
No government should have ever spent a penny on “tackling CO2”, until they had first ensured the funding was available to verify the integrity and accuracy of the temperature data. Instead excuses were given that “they are only trying their best”, when it was quite obvious that massive changes were required very quickly to improve the temperature stations.
As a result, governments committed to fund perhaps $1000,000,000,000 on the basis of data put together on a $400 PC as a sideline to other academic work. As rule of thumb, we might have suggested around 1% – 10% of the budget of $1000,000,000,000 was spent on ensuring quality temperature data. In other words between $10 to $100 billion.
Not a single institute
Whilst I talk of “an” organisation I am instinctively concerned (particularly seeing what has happened at the BBC) that “an institute”, which is in effect a monopoly could end up far worse than anything we have at the moment. Two is better, three is starts to be real competition, more would be better still. However, this would be difficult to achieve in practice – so perhaps given suitable accounting, public scrutiny and availability of data we could live with one
Not a single figure
Whilst I do not agree with those who say we cannot produce any meaningful “global temperature”, as clearly any figure is better than nothing, I do agree with those who say that such a figure is fraught with problems. As such I think any institute charged with producing a global temperature, would and should approach the same problem from various ways.
Moreover, as the raw data should be available to others, others could and should be encouraged to produce competing metrics.
As such there would not be a single figure which is THE global temperature, but instead policy makers will be given a range of figures from competing organisations.
ScottishSceptic 
The first step would be for you to actually read ISO9000.
the second step would be for you to actually identify the customers who would drive the requirements.
Here is a hint. you are not the customer.
A nice surprise seeing you here.
And yes I know ISO9001 well because as a quality manager I got the company through ISO9000.
And yes the concept of “customer” is an interesting one when it comes to global temperature. What is most important about the concept of “the customer” is to prevent internal focus to the organisation – in other words to stop the organisation being driven by what it thinks it should supply rather than what is needed.
In terms of a global temperature, it would be safer to say “customers”: the climate modellers would be one, governments another, the public a third, industry and commerce of fourth.
You must understand that Mosher believes he is the keeper of all knowledge. He is the high priest and regards any other being as inferior to his intellect. He is all knowing and has the proper response to any question or comment because as you have witnessed, they are inferior and he is compelled to show them as not worthy. He is, after all, Mosher.
No, we’re absolutely not the customer. We quite like the lack of quality control in climate data 😀 Makes it easy to paint climate science as the work of a bunch of sloppy boffins who care more about their computers than reality. Makes it easier to paint those who swear by it as either biased or deluded. Easy to set that seed of doubt that incredibly complex research and models are probably not that good if they’re based on data that’s just ‘ball park’ and not ‘high quality’. Who’d think global warming was important when the science has less formal regulation than your average business. Oh I forget, we’re supposed to take scientists on trust. Pull the other leg, it’s got bells on.
I was struck by a comment yesterday on this subject:
“I am an actuary, not a climate scientist, but it seems to me if you want to understand temperature changes, you should analyze temperature changes, not temperatures.” David in Cal
It is in fact much simpler and avoids all the data manipulation to do as David suggests, and as Motl has already done: from the raw data calculate individual station trends and compare the slopes to see larger-scale trends. This is exactly analyzing the changes, not attempting to average and homogenize the temperatures themselves.
Not only does this approach maintain the integrity of the historical record, it also facilitates what policy makers desperately need: climate outlooks for specific jurisdictions. Since the analysis is bottom-up, microclimate trends can be compiled together for any desired scope: municipal, district, region, province, nation continent.
Of course, this is not the approach of the climate establishment because it reveals the equivocal nature of the data available.
As I have commented elsewhere:
In July 2011, Lubos Motl did a similar analysis of HADCRUT3. He worked with the raw data from 5000+ stations with an average history of 77 years. He calculated for each station the trend for each month of the year over the station lifetime. The results are revealing. The average station had a warming trend of +0.75C/century +/- 2.35C/century. That value is similar to other GMT calculations, but the variability shows how much homogenization there has been. In fact 30% of the 5000+ locations experienced cooling trends.
What significance can be claimed for an average trend of 0.75C/century when the standard deviation is 3 times the size?
Conclusions:
“If the rate of the warming in the coming 77 years or so were analogous to the previous 77 years, a given place XY would still have a 30% probability that it will cool down – judging by the linear regression – in those future 77 years! However, it’s also conceivable that the noise is so substantial and the sensitivity is so low that once the weather stations add 100 years to their record, 70% of them will actually show a cooling trend.
Isn’t it remarkable? There is nothing “global” about the warming we have seen in the recent century or so.The warming vs cooling depends on the place (as well as the month, as I mentioned) and the warming places only have a 2-to-1 majority while the cooling places are a sizable minority.
Of course, if you calculate the change of the global mean temperature, you get a positive sign – you had to get one of the signs because the exact zero result is infinitely unlikely. But the actual change of the global mean temperature in the last 77 years (in average) is so tiny that the place-dependent noise still safely beats the “global warming trend”, yielding an ambiguous sign of the temperature trend that depends on the place.”
http://motls.blogspot.ca/2011/07/hadcrut3-30-of-stations-recorded.html
Thanks, that’s a really interesting post. I agree about the temperature change rather than trend. I had been thinking of trying something similar myself, but never knew where to get the data (and other things always seem more important).
The : “+0.75C/century +/- 2.35C/century”, is a very useful figure. I take it the 2.35C is standard deviation, which gives a 3SD of +/- 7C that is massive!
In no sense does this come across in their figures.
This is the problem with climate — whatever we look at — it’s always much worse than we think, just not in the way they mean it!
In CRUTem4 (for 144 land stations with temperature records starting before 1844) the only stations with a cooling trend are :-
Pamplemousses, Mauritius -1.38 C/century
Wahnsdorf, Germany -2.51 C/century
Charleston, South Carolina -0.23 C/century
Rio de Janeiro, Brazil -0.78 C/century
Cornell University, Ithaca NY -0.20 C/century.
The average of the 144 trends is 0.63 C/century; but if you plot the average temperature for each year, the trend in average temperature is 0.21 C/century from 1702.
The higher percentage of cooling trends in HadCRUT3 could be due to including sea based stations, and / or to including more temperature records that include the cooling period of 1944-1956 (-3.05 C/century) and were terminated before the warming period of 1976-1998 (+3.43/century)
Following on what Ron C said, I can’t myself see much point in “average temperature” figures. However the 0.63C/century does sound credible.
That to me is the beauty of Motl’s technique–there is no averaging at all at the level of temperatures. Thus no need for adjustments, anomalies, homogenization or infilling. You just calculate the linear trend of a station for a calendar month over its entire lifetime, be it 40, or 100, or 200 years. You get twelve monthly trends for each station. It’s the slopes that are averaged, not the temperatures. At the end, you have a trend for the average station trend, and you have the statistical distribution to see how representative of the entire set of stations is that average station.
How many times have we heard Mosher say that a set of 50 or so proper stations would give you the global reality? Well, here we have 5000+ data points, and I think the picture is pretty clear. It is just not conducive to warmist narratives.
I forgot to mention that, of course you must determine that the stations in your dataset are active in the same timeframe. For HADCRUT3 it turns out that the average station has a 77 year history, with the great majority of them 40 to 110 years old.
That’s almost never true over any significant length of time to determine variations, so it’s better to examine stations with the most coverage individually.
These Climate McScientists seriously object to being held to account, don’t they?
This is my favourite quote from that piece: “that critical analysis should start out from a position of assuming good faith and with an understanding of what exactly has been done.”
“Hey, trust us, we’re climate scientists”!
Yes, given the shining example of “Mike’s ‘Nature’ trick”, and having read HARRY_READ_ME.TXT, plus the rest of the Climategate stuff, I think not!
As for “they’re doing their best”…
I’m really really glad these jokers aren’t designing aeroplanes.
The quote which sums it up is Phil Jone’s “why should I give you the data – you only want to find fault with it”.
In other words, he simply could not conceive that it was a good thing to have his work checked out.
What anyone who really wanted to know would have said is: “you want to have a look at the data, that’s fantastic, I would really appreciate someone else having a look. What can I do to help?
Instead, anyone even asking for data was treated as an adversary.
Yes, that amazed me when I read that (in Steven Mosher’s book). I had always previously thought that science was all about scientists trying to find fault in each other’s work to ensure the integrity of the science. I can’t imagine Newton or Kepler saying anything like that.
… she tried to excuse the appalling culture within those groups producing a “global temperature figure” by saying “They are doing their best”.
I get angered at such blatant heifer dust as this myself. The simple fact of the matter is that the alarmists are falsifying the data on a daily basis. There is only intent to show catastrophic global warming — the truth is to be hidden and not revealed. And of course Mosher shows up to claim that all is well. Nothing to see here … move along now.
The only way we could get a half way decent estimate of what temps have done over the last 100 years (longer would be better of course) is to look at long established reporting stations that are outside of urban areas. A good start would be to cover just the mainland US with well sited stations; and report the raw data of the 100 or so best ones available. That and total transparency of all data, methods, and decisions.
But the government and its “scientist” minions don’t really want the truth now do they?
What angers me most, is that if I were doing this, then I couldn’t conceive of doing it without validating each and every station at least yearly.
If I took over this, I would be embarrassed by someone like Anthony Watts – not because he was checking the sites, but because a bunch of amateurs with no money had found problems that I had not already found and documented in detail.
I’d be embarrassed if I couldn’t tell Anthony every single station, its quality and a corrective plan to correct any problems – and that plan would have to have a detailed recalibration of the site in order to assess the size of the error.
But no!!! Their attitude is “none of our business to put problems right”.
WELL WHO’S BUSINESS IS IT?
It’s reasonable to say that the temperature data is their core business. While it’s true that we’re not climate science’s customer, we’re the customer’s customer. Governments want to know why the public are having a hard time buying what they’re selling.
Climategate was forgiveable. Weird, given how long CAGW has be on the agenda but forgiveable because they’d never come up against real scrutiny before, but now? Almost five years later?
If you can prove the US temperature system has flaws, what can you say about the rest of the World?