Forgetting for the moment the various credible accusations based of their emails of distorting data, tampering with the peer-review process, conspiring to hide information from FOI, etc, there is the repeated exscuse that they could not store all their data in the 80s.
Credible in your opinion
Never a good start to try and be reasonable if you are going to load your very first sentence like that. You seem them as credible, I don't see such carefully selected, out of context snippets as credible.
I see them as credible, since they'd already been accused prior to the whole incident of most of the things that in those emails we find evidence of. Call it a few snippets, but how many emails on the matter are necessary? IF I conspire by email to commit murder over the course of a couple years, and I average around 1000 emails a month, there will be somewhere around 36,000 emails over 3 years with probably only a few involved with the murder, none of which are likely to say "Yes, we shall commit premeditated murder, and this is absolutely not a joke, on Person X"
This, of course, is utter BS. By 1974 IBM gave us nice little cylinders you could hold in your hand that stored 50 MB, we had 3 1/2 drives storing a quarter of a Mb by 82 and tape drives storing 200 MB with rapid access by 84 that were slightly smaller than a paperback - and not a thick fantasy novel.
Now how much data was there, that they felt obliged to toss it away? We're spoiled by videos taking up gigabytes but a one megabyte is a lot of memory where data is concerned.
If they stored it in ASCII, as one byte per character, or in decimal at 4 bit instead, with just numbers 0-9 and maybe a break character or two (16 to play with at 4 bit after all) then one megabyte lets them store either one million characters or 2 millon numerical digits.
What was that data? Was it just high and low for the day, maybe a four digit number in celsius? Some abbreviated timestamp and four digits then, with a longer timestamp from time to time and a location identifier? You could record a hundred years of data on 1 megabyte from one monitor station that way and still have room left over.
But maybe they did more, maybe they took hourly readings of temperature, pressure, humidity, wind direction and speed, rain fall. You wouldn't record that in an easy read format, you'd mash it, so maybe 176928161320145 means 17.69 Celsius, 28.1 pressure, 61% humidity, wind at 320 bearing 14.5 km/h. Toss in a timestamp like 840011445 for 001 - January 1st of 1984 and 1445 or 2:45 PM, what was that, 25 characters, 25 bytes, 26 with a break or more likely 13 since you'd not waste memory with ASCII. At hourly readings, about third of a kilobyte a day per station, about 100 kb a year, per station, hourly readings of everything I can think of with no timestamp abridgement and probably more significant digits then were actually recorded, but maybe they did cloud cover percent, dewpoint, etc too. So let's say 100 kb, keeping in mind that a 400k word book like RJ writes contains millions of characters, so one stations hard copy results for a year could be printed in less than a paperbakc, much less, even with nice formatting. One of those 50 Mb cylinders from 1974 shold have been able to store a decade of data from 50 locations, one of those tape drives from 1984 could have done 200 and fit in less space than a paperback. How many monitoring stations did they have, ten thousand? All 10,000, for a decade, 10 Gig, neatly fitting on to 200 3480 tapes or 800 of the old 3850 ones from 1974. And this whole time, a sdata is coming in, the clutter isn't happening because more compact and cheaper memory devices are arriving faster than you're filling your old ones up
So no, I don't buy the memory limitation stuff some people have been chucking around.
You don't get rid of your raw data, especially when it can't be easily replicated in a lab. Pretty hard to repeat weather on earth. Sure, you've got a neat experiment that can be done over quick and easy and shows no anomalies, you don't really need to keep the data around, but even then you do becuase someone might challenge it, and some enterprising student might come by years later, notice an anomaly and find a whole new field of science from it. "Interesting, your temps are .01 degree to high in this region, even a bit higher at night, turns the ground is warmer than it shoud be because you're sitting on a patch of uranium. Weird, holy heck, it's an asteroid impact! And we dug it up and found out that it dates to an extinction event!" etc. Vast amounts of new science emerges from what often looks and essentially was noise, totally irrelevant often to the experiment being run but a vital clue for something else. SO them losing their raw data is bad just for that.
It also makes all their results that rely on that data garbage, probably true anyway, but still garbage, because no one can look at the data to check. It's not just a bunch of anti-GW people pissed at this, it's a bunch of scientists, many firm advocates of AGW. And we know about this without those emails, we've known for months, probably why they got hacked. Dr Pielke - climatoligist and big advocate of AGW - asked for their data months ago and posted their BS answer along with some very nasty remarks.
I'd not seen about this before, and it does reflect poorly on them as it stands. Though I think you are hugely over estimating public sector IT provision in the UK - as someone who works in the public sector and knows a fair number of other people who do, I wouldn't bet against some places not having that level of storage now
Probably those NMS and met stations did not. They probably also have centrlaized locations they report up to, so it isn't necessarily a disaster. But the problem is, the data now has to be checked, politics will require that. If someone goes and recollects all that data from the stations or whatever regional central storage they have, and runs it through the processes they did, and gets the same result, then all is well on that front and they get off with the academic equivalent of a slap on the wrist for piss poor procedure. Normally someone wanting to check the accuracy of their data would simply pick a few thousand random data points and contact the stations that gathered them to confirm they were right, maybe run an anomaly hunter program and check those points too. Now, to confirm any of their results, it all has to be repeated, and any error will be seized on by their political opposition. A difference in result of even a small amount will be used as a trumpet for decades. Personally, I doubt they tampered with their data, when you have millions or billions of data points it's hard to tamper with it directly in a fashion that doesn't get noticed, if you increase/decrease a number a lot, it pops up as an anomaly, if you screw with a whole bunch just a little someone notices when they randomly sample your data for comparison. Anything else, with so much data, shouldn't have much of an effect.
No, the way people screw with the results is with the method. Take this example, let's say you get data in both fahrenheit and celsius, each to two decimal places, but you only want three. If you round those fahrenheit numbers before converting them to celsius you will get a skew, since fahrenehit is 5/9 of celsius. This sort of thing can happen on accident too, which is why you need to keep the data you used. It's sort of like chain of evidence, if you have a piece of evidence pass through a dozen hands, and eleven nicely signed transfer forms, you're fine, if it's only ten and one is missing or unsigned, you have a problem. Now, this all goes away if say, there are a handful of central archives for their data and those can all produce it neatly and quickly and when processed through their method the same numbers result. I'm guessing this isn't the case though or it would have been done by now, data all being digitized and such, the transfer time should be measured in hours, or maybe a week if the archives or monitors need to send a hard copy, like a CD.
Having looked this up on the internet now though, it seems that it isn't a case of them having thrown away the only copies of thay data. The data was provided to them by local Met Offices around the world and is being stored by those Met Offices locally still (which kinda undermines a nice size paragraph of yours). They also claimed that the data they got rid of was when they moved building and didn't have space to store the paper and tapes it was stored on, which kind of skews your estimates, without knowing the volume of paper involved (or how much data was on each tape and the total humber of tapes).
). They also claimed that the data they got rid of was when they moved building and didn't have space to store the paper and tapes it was stored on, which kind of skews your estimates, without knowing the volume of paper involved (or how much data was on each tape and the total humber of tapes).They didn't just have data from Met IIRC, and it undermines nothing. You don't rely on others to keep your data for you. 'Earthquake hits Australia' headlines the day your data from that place gets lost because a gas main snapped and the building got torched. Not having your own copy of the data you are using is sort of like saying 'Driving can be dangerous, with many injuries, that's why I always keep a first aid kit... under my bathroom sink' Transcription errors occur, data gets corrupted, if someone reproduces your results from the original and say 'what the f?' It's nice to be able to show them your data that you used so when fingers start getting pointed someone can notice 'Hey, the station has the temp for that day at 23.76, yours says 27.36' instead of 'how do you account for this particular data average being two percent higher in yours compared to ours?' Which is really not a situation you want to be in when your field of study is a known political firestorm. Again, chain of evidence.
Still, while not what you seem set on building it into (as they deleted their copies of the original data, not the original data), it is dubious behaviour at best by them.
What do you think I'm trying to build this into? I haven't accused them of fudging their data deliberatly, I've accused them of engaging in sloppy science which opens a floodgate for people to attack AGW on. Maybe you've missed all the posts I've made in the last two years on AGW where I say things like 'While I agree the evidence indicates warming, and man made warming, I...' and so on.
Now as for the emails, yeah, it was an invasion of privacy, though we now know they were conspiring - out of their own mouths - to avoid FOI, which is contemptible from a scientist. Just because it was obtained immorally doesn't mean we stick our heads in the sand, if someone is peeping in people's bedroom windows and sees some guy with a ten year old tied up on his bed, we don't go "Oh, well you invaded his privacy".
I think you rather miss the point - it is this. When someone (or ones) illegally obtains informtation then releases it at a carefully timed point to further a political end (and I consider timing it to try and influence the Copenhagen talks a political end, given it is now known that UEA was hacked several months ago) then releases their highlights of it, perhaps the right thing to do isn't to dance to their tune unquestioningly, like puppets.
No, I think you miss the point, the motivations of the opposition are irrelevant, we already know that on both sides of the debate there are tons of people who don't really know what they are talking about and walk around with their fingers in their ears. There ar epeople who would deny AGW if in 2030 someone opened up a banana plantation in Alaska, and there are people who would claim AGW was still true if the CO2 level hit 10% and the temperature remained the same. I don't care about thos epeople, even if they are on the 'right side', they'll still morons. They just won the coin toss when it came to picking which side they'd blindly ally themselves too. You support AGW, so any action against it is one you view in a bad light. I suspect you might dance to a different tune yourself if it was action against something you were against.
And seriously, you are trying to compare this to paedophilia?
Again, I assume you've read enough of my posts to know I have a penchant for using extreme examples. However, to push the point, don't be too surprised if you hear someone say 'they molested the scientific process'
No, of course I don't compare their action to pedophilia, you are putting hostile spin on my comments right after saying we should put a good spin on theirs. The use of the term pedophile wasn't a smear on them, it was blatantly obvious from the context, or should have been, that I was using pedophilia as an example of extreme behavior that overrides summarily dismissing it because the source was criminal themselves. I'm not proposing hanging someone because our lone evidence is a peeping tom, anymore than I propose blackballing these guys just becaus eof the emails. The peeper is justification for a warrant, not a lynch mob.
Very classy and reasoned, no attempt to load the end of your long post the way you did the start then, I see. It is a pretty stupid comparison but one that is so uncalled and untasteful for I'm not even going to bother picking it apart.
Snoop, please don't stick ideas in my head then go off analyzing them. There is only an untasteful comparison there if you stick it in. Your reaction through out this email smack of denial, remember that I am a scientist and climatology is viewed as a sub-discpline of my own field. Remember that I am a AGW supporter. You react to everyone one of my comments like I don't know what I'm talking about and will simply seize any excuse to discredit the whole field, when I do know what I'm talking about, better than most on this board by far, and I usually don't make a habit of being in denial of something I actaully happen to believe in. This is like you calling me a holocaust denier if I said 'The study showing 6 million dead at Auschwitz may no longer be reliable because the original data collected by Dr X who came up with it can't be located'
The intuitive mind is a sacred gift and the rational mind is a faithful servant. We have created a society that honors the servant and has forgotten the gift.
- Albert Einstein
King of Cairhien 20-7-2
Chancellor of the Landsraad, Archduke of Is'Mod
- Albert Einstein
King of Cairhien 20-7-2
Chancellor of the Landsraad, Archduke of Is'Mod
Global Warming: Climategate
- 28/11/2009 05:42:50 PM
704 Views
I think the problem is that it just isn't actually much of a story, though clearly some wish it was
- 29/11/2009 11:49:19 PM
445 Views
That dog don't hunt
- 30/11/2009 07:49:04 AM
368 Views
What you do or don't do with your dog is between you and your dog. And perhaps the legal system.
- 01/12/2009 04:24:00 PM
407 Views
The loss of the data was known prior to the emails
- 01/12/2009 07:39:36 PM
477 Views
Sounds to me like you're going deep into your playbook for excuses and sidesteps.
- 02/12/2009 08:03:51 PM
371 Views