Climate Audit

[TCO]

http://www.climateaudit.org/

I turned from like liking that site and respecting the guy who runs it, to not liking it or respecting the guy.

Actually then and now I think the algorithms and systems are intersting. And that the guy does interesting work replicating things, writes his own code (and is in his 60s) and has read a lot of literature.

But I found that as I probed the guy, he was poor at answering tough questions. BAsically was not interested in running critical tests, but just in finding snippets, running with them and playing to his little blog hoi polloi. Guy has not published a paper in 4 years...and really only has one "real" paper to his name. A Geophysical REview Letter. He claims to have 5 publications, but two of those are actually replies to comments from his GRL paper. And the other two are in Energy and Environment, which is a "skeptic journal" that is not abstracted by normal science services.

[KrazyHorse]

How do you feel about temp proxies (specifically, which are more reliable, which less in your view)?

[TCO]

Originally posted by KrazyHorse
How do you feel about temp proxies (specifically, which are more reliable, which less in your view)?

Kitty: I tend to contribute to these areas, not so much from researching things and being knowledgeable, but just asking logical questions in a sort of annoying nuke officer/Socratic method. That said, my impressions (only) on the different proxies:

Coral: Really, really good proxy. You can look at "wiggle matching" and see that the coral really does a great job of pretty much "tit for tat" following instrumented changes year by year. Also, it has "annual resolution". Unfortunately, you have to be near coral.

Trees: Large availability, "annual resolution". There are confounding factors, however that does not mean all info is lost. See the wiki entry on dendroclimatology (which I wrote, from memory so as not to plagiarize, but likely influenced by the Ultimate Tree Ring site or whatever it's called). Things like limited stand (treeline) can help isolate the factor of concern. That said, I think there is a lot of "mess" with the samples that have been taken in terms of sampling method not determined beforehand, opportunistic use of various chronologies not developed for temp sensing etc. Pouring a bunch of money and doing some real government based fundamental work here would benefit the field.

Boreholes: Don't get annual or decadal resolution. And only goes back a few centuries, but is pretty close to a "direct" measurement. Some danger of confounding from water flows. Also the equations for modelling require some good characterization of thermal conductivity as a function of depth. I don't really know enough to judge it, other than that it's astonishing that people can know what temp it was a hundred years ago from drilling a hole.

Icecores: I think they look at O-18. Rain out is a confounding factor. More effective in arctics than near equator.

Speleo: Don't know much about it.

Sediments: Don't know much about it. Think it can be effective.

Documentary: I think this is very worthwhile to look into (harvest dates and the like). Limit to what you can get of course.

Instrumental: Sometimes they use "old temp measurements". These of course are stellar compared to tree rings or the like. Really near quality of current instruments given problem you are trying to do.

--------------------

I think it would be nice to look for other proxies. I heard a proposal once for something that was like a "second order" isotope based proxy. The guy wanted some money to develop the method. I think his idea kinda crapped out...but innovation like that should be encouraged.

[TCO]

Here's an example of an annoying post:

http://www.climateaudit.org/?p=4449

He's got so much extraneous name calling, other previous debates, that it's actually destracting to try to understand exactly what he's alleging. And although he's on a website, he blows off using hyperlinks to identify relevant (as opposed to non sequitor blathering) previous issues.

[TCO]

Here's my try at cleaning up his post:

First the unedited version (figures not duplicated):

Replication Problems: Mannian Verification Stats
by Steve McIntyre on November 25th, 2008
If anyone feels like sticking needles in their eyes, I'd appreciate assistance in trying to figure out Mannian verification statistics. Even when Mann posts up his code, replication is never easy since they never bothered to ensure that the frigging code works. Or maybe they checked to see that it didn't work. UC's first post on the matter wondered where the file c:\scozztemann3\newtemp\nhhinfxxxhad was. We still have no idea. This file is referred to in the horrendously written verification stats program and it may be relevant.

With UC's help, I've been able to replicate quite a bit of the CPS program (the EIV module remains a mystery.)

I've been testing verification stats with the SH iHAD reconstruction. I mentioned previously that Mannian splicing does not always use larger proxy networks if they get "better" RE stats with fewer proxies. This Mannian piece of cherry picking is justified in the name of avoiding "overfitting" although it is actually just the opposite. It reminds me of the wonderful quote from Esper 2003 (discussed here):

this does not mean that one could not improve a chronology by reducing the number of series used if the purpose of removing samples is to enhance a desired signal. The ability to pick and choose which samples to use is an advantage unique to dendroclimatology.
Mining promoters would like a similar advantage, but, for some reason, securities commissions require mining promoters to disclose all their results.

Mann's reconstruction archive in 2008, as with MBH98, only shows spliced versions - some habits never change, I guess. But in the SH iHAD run, the AD1000 network remains in use right through to the 20th century, with all proxies starting later than AD1000 being ignored - all in the name of not "overfitting". But the long run of values from a consistent network is very handy for benchmarking and, with much help from UC's Matlab runs, I've managed to very closely replicate the SH iHAD reconstruction from first principles, as shown below - this graphic compares a version archived at Mann's FTP site with my emulation.


For comparison, here is an excerpt from Mann SI Figure S5d (page 11), which has an identical appearance.


You can upload an original digital version of this reconstruction (1000-1995) as follows:

url="http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/data/reconstructions/cps/SH_had.csv"
had=read.csv(url)
temp=!is.na(had[,2])
estimate=ts(had[temp,2],start=min(had[temp,1]))
A digital version of the "target" instrumental is also at Mann's website and can be downloaded as follows:

url="http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/data/instrument/iHAD_SH_reform"
target=read.table(url)
target=ts(target[,2],start=1850,end=1995)
The reported verification statistics for the SH iHAD reconstruction are also archived and can be downloaded as follows (load the package indicated). BTW this is a nice package for reading Excel sheets into R.

library(xlsReadWrite)
url="http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/cps-validation.xls"
download.file(url,"temp.xls",mode="wb")
test=read.xls( "temp.xls",colNames = TRUE,sheet = 14,type = "data.frame",colClasses="numeric")
count=apply(!is.na(test),2,sum);count
temp=!is.na(test[,1])
stat=test[temp,!(count==0)]
name1=c("century", c(t( outer(c("early","late","average","adjusted"),c("RE","CE","r2"), function(x,y) paste(x,y,sep="_") ) )) )
names(stat)=name1[1:ncol(stat)]
stat[stat$century==1000,]
# century early_RE early_CE early_r2 late_RE late_CE late_r2 average_RE average_CE average_r2 adjusted_RE adjusted_CE adjusted_r2
# 1000 0.0746 -1.663 0.3552 0.7194 0.1475 0.303 0.397 -0.758 0.3291 0.397 -0.758 0.3291
Given digital versions of the reconstruction and the "target", it should be simplicity itself to obtain standard dendro verification statistics. But, hey, this is hardcore Team. First, Mann does some Mannian smoothing of the instrumental target. Well, we've managed to replicate Mannian smoothing and can follow him through this briar patch.

library(signal) # used for smoothing and must be installed
source("http://www.climateaudit.org/scripts/mann.2008/utilities.txt")
cutfreq=.1;ipts=10 #ipts set as 10 in Mann lowpass
bf=butter(ipts,2*cutfreq,"low");npad=1/(2*cutfreq);npad
smooth=ts( mannsmooth(target,M=npad,bwf=bf ) ,start=1850)
Now the "early miss" verification stats using a simple (and well-tested) program to do the calculations:

verification.stats(estimator=estimate,observed=smo oth,calibration=c(1896,1995),verification=c(1850,1 895))[c(2,5,4)]
# RE.ver CE R2.ver
# 0.2859 -0.9142 0.1528
And for the "late-miss" stats:

verification.stats(estimator=estimate,observed=smo oth,calibration=c(1850,1949),verification=c(1950,1 995))[c(2,5,4)]
# RE.ver CE R2.ver
# 0.8046 0.4111 0.455
These should match the early_ and late_ values, but don't. The inability to replicate the r2 values is particularly troubling, since these are not affected by the various scaling transformations. I simply haven't been able to get the reported verification r2 values using many permutations.

Since the reconstruction ties together both to digital and graphic versions, perhaps the archived instrumental version https://www.meteo.psu.edu/~mann/supp...iHAD_SH_reform is not the same as the c:\scozztemann3\newtemp\shhinfxxxhad .

The code for the verification stats is at
https://www.meteo.psu.edu/~mann/supp...i1950_1995sm.m and https://www.meteo.psu.edu/~mann/supp...1850_1895sm.m. They seem to have learned their programming style from Hansen, as the code is replete with steps that don't seem to have any function, unhelpful comments made less helpful in places by inaccuracy and, most of all, by an almost total lack of mathematical understanding and organization in implementing the code.

[TCO]

The cleaned up post:

Replication Problems: Mannian Verification Stats
by Steve McIntyre on November 25th, 2008

Please help me figure out Mann’s verification statistics in MannO8 [[ref]].

I can’t replicate the work, because the code does note work. UC [[hyperlink]] asked where the file c:\scozztemann3\newtemp\nhhinfxxxhad was. It is referred to in the verification stats program [[use exact name, is this “EIV” below?]] and may be relevant.

I've been testing verification stats with the SH iHAD reconstruction.

Mann's reconstruction archive in 2008, as with MBH98, only shows spliced versions. However, in the SH iHAD run, the AD1000 network remains in use right through to the 20th century, with all proxies starting later than AD1000 being ignored. While I have some disagreements with leaving data out of a reconstruction, the long run of values from a consistent network is very handy for benchmarking. With help from UC's Matlab runs, I've managed to almost exactly replicate the SH iHAD reconstruction, with my synthesized R code.

[[Add a proper figure caption, add proper labels within the graph, the now excised sentence before was not a suitable substitute]]

[[Excised the figure from Mann’s SI, since you don’t explain the point of the “comparison” which you indicate (in passive mood) that the reader could do. Also, that figure has 3 lines in it and you don’t say which is supposed to correspond to your simulation. Basically, you just drop that picture like a turd without explaining the point of it.]]

You can upload an original digital version of this reconstruction (1000-1995) as follows:

url="http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/data/reconstructions/cps/SH_had.csv"
had=read.csv(url)
temp=!is.na(had[,2])
estimate=ts(had[temp,2],start=min(had[temp,1]))
A digital version of the "target" instrumental is also at Mann's website and can be downloaded as follows:

url="http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/data/instrument/iHAD_SH_reform"
target=read.table(url)
target=ts(target[,2],start=1850,end=1995)

The reported verification statistics for the SH iHAD reconstruction are also archived and can be downloaded as follows (load the package indicated). BTW this is a nice package for reading Excel sheets into R.

library(xlsReadWrite)
url="http://www.meteo.psu.edu/~mann/supplements/MultiproxyMeans07/cps-validation.xls"
download.file(url,"temp.xls",mode="wb")
test=read.xls( "temp.xls",colNames = TRUE,sheet = 14,type = "data.frame",colClasses="numeric")
count=apply(!is.na(test),2,sum);count
temp=!is.na(test[,1])
stat=test[temp,!(count==0)]
name1=c("century", c(t( outer(c("early","late","average","adjusted"),c("RE","CE","r2"), function(x,y) paste(x,y,sep="_") ) )) )
names(stat)=name1[1:ncol(stat)]
stat[stat$century==1000,]
# century early_RE early_CE early_r2 late_RE late_CE late_r2 average_RE average_CE average_r2 adjusted_RE adjusted_CE adjusted_r2
# 1000 0.0746 -1.663 0.3552 0.7194 0.1475 0.303 0.397 -0.758 0.3291 0.397 -0.758 0.3291

Given digital versions of the reconstruction and the instrumental "target", it should be simple to run verification statistics. However, the reported statistics are not from doing that test, because Mann does several other operations on the curves [[so why not do it yourself and report the results, sans intermediate manipulations?]]

First, Mann does some Mannian (see below for method) smoothing of the instrumental target.
library(signal) # used for smoothing and must be installed
source("http://www.climateaudit.org/scripts/mann.2008/utilities.txt")
cutfreq=.1;ipts=10 #ipts set as 10 in Mann lowpass
bf=butter(ipts,2*cutfreq,"low");npad=1/(2*cutfreq);npad
smooth=ts( mannsmooth(target,M=npad,bwf=bf ) ,start=1850)

Now the "early miss" verification stats are calculated:

verification.stats(estimator=estimate,observed=smo oth,calibration=c(1896,1995),verification=c(1850,1 895))[c(2,5,4)]
# RE.ver CE R2.ver
# 0.2859 -0.9142 0.1528
And for the "late-miss" stats:
verification.stats(estimator=estimate,observed=smo oth,calibration=c(1850,1949),verification=c(1950,1 995))[c(2,5,4)]
# RE.ver CE R2.ver
# 0.8046 0.4111 0.455

These should match the early_ and late_ values, but don't. My inability to replicate the r2 values bothers me, since these are not affected by the various scaling transformations. I simply haven't been able to get the reported verification r2 values using many permutations [[describe]].

Perhaps the archived instrumental version https://www.meteo.psu.edu/~mann/supp...iHAD_SH_reform is not required reference for analysis.

The code for the verification stats is at
https://www.meteo.psu.edu/~mann/supp...i1950_1995sm.m and https://www.meteo.psu.edu/~mann/supp...1850_1895sm.m.

The code is difficult to understand as there are functionless steps, incorrect comments, and poor organization (mathematic simplifications could reduce the size).

[Traianvs]

Dendro, arctic icecores and to a lesser degree sediment deposits (in Scandinavian lakes for example) are fantastic. Like everything you've got to interpret the data correctly. Still they are a godsend for archaeologists in reconstructing climate and the environment in general. Hooray to dendro-calibration for C14 dating!

[TCO]

Originally posted by Traianvs
Dendro, arctic icecores and to a lesser degree sediment deposits (in Scandinavian lakes for example) are fantastic. Like everything you've got to interpret the data correctly. Still they are a godsend for archaeologists in reconstructing climate and the environment in general. Hooray to dendro-calibration for C14 dating!

1. Dendrochronology (wiggle matching) is fascinating. What work do you do here?

2. I do think that dendroclimatology is not quite as simple or iron clad as dendrochronology (just dating things, vice inferring climate).

3. I basically wrote this article: http://en.wikipedia.org/wiki/Dendroclimatology

[Traianvs]

Originally posted by TCO


1. Dendrochronology (wiggle matching) is fascinating. What work do you do here?

2. I do think that dendroclimatology is not quite as simple or iron clad as dendrochronology (just dating things, vice inferring climate).

3. I basically wrote this article: http://en.wikipedia.org/wiki/Dendroclimatology

Just read it. You're one of the good people on this earth writing on Wikipedia, while I am one of those leechers that take all that effort for granted

I studied archaeology, maybe a bit obvious from my post. Dendrochronology has become very very important over the past years, causing a good number of relative dating sequences to be altered/enhanced. Unfortunately we were only taught the basics, so we could at least understand the literature of those damned scientists.

Actually dendroclimatology is just one of the many roads towards the reconstruction of the environments we study. Palynology, paleobotany, archaeozoology, geoarchaeology, paleogeomorphology to name a few disciplines. Or chemistry, like strontium analysis in the bones of prehistoric people can be very useful. Impossible to get to know all of it, so we only get the basics so we can interpret it right. We hire real scientists to do the actual job

[TCO]

so you are a glorified ditch digger?

[Traianvs]

No disrespect to ditch diggers from me!

[TCO]

Here's another CA post, pissing me off:

http://www.climateaudit.org/?p=4465

1. Actually, this post is simpler and clearer than the majority on CA, but still contains a lot of flaws.

2. Pompous language: "In our examination of the new Mann proxies, there is a notable increase in the prevalence of speleothem proxies in the MWP network. "

Why not, instead: "Mann's new study contains more MWP speleothelm proxies than in MBH99."

(And giving the numbers would giver more content. And actually prevelance of speleothelms is not even relevant to the point later in the post, which is about calibration.)

3. Bunch of extranous hoi polloi crap: chimpanzee reference, mote in eye reference.

4. Implicitly the post seems to lump Gavin together with Mann and Loehle together with CA. However, this is not the right way to look at things. And Gavin sjhould be capable of finding a fault in Mann and McI one in Loehle. Although Gavin also seems to adopt this "us versus them", and "don't say bad stuff about my side" attitude like McI did with Loehle's abortion of a published paper (where he failed to call out specific faults or even that it was flawed and used a Clintonism instead "similar to Moburg".)

5. Fails to actually link to or describe the degree of correlation of the Mann speleothelm, when this is at the heart of believing the orientation (instead just citing the "archive" in general terms...expecting us to go dig for the key numbers in science terms in considering the series.)

6. Misleadingly quotes one paragraph of two from Gavin, which makes it appear that Gavin only thinks speleothelms should be oriented one way, when Gavin's earlier paragraph was much more thoughtful, discussing the length of instrument run for calibration, as well as implicit dating inaccuracy in speleothelms. This selective quoting is either stupid or dishonest, possibly both. Although having the link does allow us to conveniently find the skewed take on Gavin's original comments.

7. Totally lacks any real science insight. Does SM think the speleothelm here should be put the opposite way (would require looking at the quality of the calibration, the actual numbers, but SM blew that off...or hid it since it didn't help the "gotcha".) Does SM think that speleothelms should only be calibrated in one orientation (and if so, how to handle those proxies which don't show that orientation)? How to assign a magnitude of implied temp response? Note that this is actually a more general issue for all proxies. SM has touched on it, but mostly for editorial games...not to try to actually think about it and make an insight.

[Berzerker]

Icecores: I think they look at O-18. Rain out is a confounding factor. More effective in arctics than near equator.

What is rain out?

[SlowwHand]

TCO. I think it finally happened. You done lost your mind.
We're talking totally. The days of walking the edge are over. You have plunged into the abyss.

[TCO]

Originally posted by Berzerker


What is rain out?

http://www.nationmaster.com/encyclopedia/Oxygen-isotope-ratio-cycle

(also do a search on climate audit).

Basically O18 is heavier than O16, thus is preferentially a part of precipiation. At lower temps, O18 is more present (it's heavier and at lower temps all molecules hjave less energy and this means difference mroe important, at higher temps the reverse).

Above para shows how basic O18 concentration works as a proxy, but if there is prolonged rain/snow, the O18 concentration can get "rained out" to the extent that O16 ends up just being only thing left and thus confounding the interpretations.

Capisce?