A New Puzzle: Two Versions of the Sommer Report

A recent David Holland FOI has turned up an astonishing new riddle about the relationship between UEA and the Muir Russell panel: there are two different versions of the Sommer Report on the Backup Server, both dated 17 May 2010 and both entitled “UEA – CRU Review: Initial Report and commentary on email examination”. One version was included in the Muir Russell archive of online evidence – see here – it was only two pages long. A different 10-page version was produced by UEA in response to David Holland’s FOI – see documentation or here as html. The longer version contains details not included in the (apparently) expurgated version published by Muir Russell. The short version is derived from the longer version. Although the two versions of the report are both said to have the same author and bear the same date, there are differences in formatting that, in my opinion, point strongly to the shorter version having been prepared by someone other than Peter Sommer for reasons that, at present, are not entirely clear. If, on the other hand, Sommer himself did prepare the shorter version as well as the longer version, the UEA appears to have withheld correspondence documenting their reason for requesting a second version of the report and whether the second version was backdated.

As a prefatory comment, I urge CA readers to purchase Andrew Montford’s excellent “Hiding the Decline” for a thorough discussion of the “inquiries” that failed to inquire, written in Andrew’s usual lucid style. I had seen a pdf draft; Andrew kindly sent me a real copy. It read even better as a paperback back than on the computer screen. So please support Andrew – see here.

I’ve commented in the past on the very cursory analysis of the Backup Server. One loose end, recently pursued by David Holland, was that Sommer had stated (in the expurgated version here published by Muir Russell) that he had been retained by the UEA (rather than the “Independent Climate Change Email Review”) as follows:

“I am asked by the University of East Anglia to look at the back-ups of the computers of the key researchers in CRU as they are held on the back-up server to see if it is feasible to identify email traffic which was not publicised on the various websites, but nonetheless related to the same issues and might justify further investigation by the Independent Review into the publication of the emails and the allegations of inappropriate scientific and other practice which had subsequently been made.”

Quoting the above paragraph, in November 2012, David Holland asked the UEA for its correspondence with Sommer as follows:

Please supply me electronic copies of the correspondence with Prof. Sommer in connection with this assignment including his invoice for this work.

In december 2012, the UEA returned the following documentation in one bundle:
1. a four-page contract between the UEA and Peter Sommer dated April 26, 2010
2. an administrative email dated April 27, 2010 from Elaine Rymarz of the Registrar’s Officer to Sommer;
3. an invoice dated May 26, 2010 from Sommer to UEA for 37 hours of work between April 26 and May 24, 2010
4. a ten-page report from Sommer dated May 17, 2010 entitled “UEA – CRU Review Initial Report and commentary on email examination”

It seems very likely to me that there is other correspondence between the UEA and Sommer. It’s hard to picture the contract being administered with no correspondence between the signature of the contract and delivery of the report, especially with the problems and delays in obtaining access. However, that’s a problem for a different day.

As noted above, there are some puzzling stylistic differences between the two versions (looking here at the documents in terms of form as opposed to content, just as Mosher did when he identified Gleick as the author of the forged Heartland document).

In the short Muir Russell version, the author and date at the end of the document are left-justified in the same font as the rest of the report, while in the FOI version, the author and date are right=justified. The author in the Muir Russell version is “Professor Peter Sommer”, while the author in the new version is just “Peter Sommer”. Sommer’s website was listed in the Muir Russell version, but not in the FOI version. Compare the two below.

sommer signature uea
sommer signature mr

There are other format differences. The numbered paragraphs in the Muir Russell version are left justified at the same tab as the section headings, while in the new version, they are indented. The page margins also look different.

The long version in the FOI production also looks like it was scanned from a paper copy – it looks like there are punch-holes in the left margin.

As noted above, it is my surmise that the shorter (expurgated) version was prepared by someone other than Peter Sommer, drawing on the authentic Sommer report.

The Document Properties of the Muir Russell version show that Muir Russell PDF version was produced on June 23, 2010 from a Word document by “Will” [William Hastie of the Muir Russell panel].
sommer document properties

Did Hastie prepare the Word document and if so, why? Or was the Word document provided to him and if so, by whom? Did Sommer provide UEA with a Word document or did someone else provide the Word document? If Sommer was the author of the Word document, was it produced at the request of UEA subsequent to their receipt of the longer FOI version? If so, what was the reason for UEA requesting a shorter version and why wasn’t the correspondence requesting this shorter version produced as correspondence between the parties. Did the Muir Russell panel ever receive the unexpurgated Sommer Report? If so, why didn’t they publish it?

In terms of content, the Muir Russell version is seriously expurgated from the FOI version. For example, in the FOI version, Sommer questioned whether there was a plausible basis for the police to classify the material as “Secret” and suggested that the UEA and Muir Russell ask the police to downgrade the security classification. No hint of this recommendation occurs in the expurgated Muir Russell version.

sommer on sevrecy

There are many questions about the relationship between UEA and Muir Russell. This is only one.

Update Jan 14, 2013. Some further information on the two versions of the Sommer report.

The University was sent an initial draft of Professor Sommer’s report. The University and the review team both took the view that the report was too long and would benefit from being more focused and concise. It appears that those comments were taken on board by Professor Sommer as his final report, which is publicly available on the internet, was shorter.

62 Comments

  1. Willis Eschenbach
    Posted Jan 7, 2013 at 11:29 AM | Permalink

    You gotta love these guys, a corkscrew has nothing on them. There’s not one straight story in the batch.

    As always, fascinating, my friend,

    w.

  2. Skiphil
    Posted Jan 7, 2013 at 12:12 PM | Permalink

    P.3 of the ten page (unexpurgated) version of the Sommers report seems to be missing from the PDF linked above:

    Click to access Appendix%20A%20Data%20file%20155.pdf

    I cannot view items 13 – 18 which presumably constitute p.3

    Did David Holland receive that page?

    Steve: David Holland received the documents as the linked pdf. I can see page 3 in it.

    • Skiphil
      Posted Jan 7, 2013 at 12:23 PM | Permalink

      P.s. sorry , “Sommer” with only one ‘s’

      It may be difficult to get any statement from him about what transpired, since his kind of business would seem to depend entirely upon the good graces of the clients paying for the forensic digital services. Such clients want no tales out of school, no matter how fuly justified. UEA seems to have exercised a great deal of hidden control and influence over what tasks the Muir Russell panel could carry out.

      It does seem that one effect has been to obscure just how cursory and uninquisitive the “inquiries” really were.

      Steve: Sommer’s business requires him to maintain confidentiality with customers. There is zero chance that he will respond to questions. It’s not even worth trying.

  3. Posted Jan 7, 2013 at 12:21 PM | Permalink

    I see 13-18 when I click on the link.

    What seems to have been dropped from the Sommer report are many complaints about the format of the the files and his excuses for not doing a competent job.

    Steve: Given his very limited access to the documents, there was little that Sommer could do. I’m sure that he wanted to document this and he was both right and justified in documenting this. He did not obtain access to unencrypted files until 4.30 on Friday the 14th. He was only able to have access between 9 am and 5 pm on weekdays and not allowed to run programs overnight. He was required to report on Monday the 17th. Regardless of the limitations of his report, I don’t think that it’s reasonable to say that he didn’t do a “competent” job given the restrictions and lack of access. The larger issue is why Sommer wasn’t given an opportunity to do a proper analysis.

    • Skiphil
      Posted Jan 7, 2013 at 12:28 PM | Permalink

      Strange, I’m still not getting that page even after re-loading many times. I’m viewing from my iPad, I’ll try again from a PC when I get to one, in case that makes any difference (Steve Jobs hated Adobe, did he have something against the PDF format?).

      • iskoob
        Posted Jan 7, 2013 at 12:54 PM | Permalink

        Interesting hypothesis, but Apple has nothing against PDF per se (which the Adobe corporation no longer owns, as of 2008). Far from it—the Quartz drawing engine that powers all Apple displays was based on, and designed for toll-free compatibility with, the PDF language. Which is not to say it’s easy to read a large PDF document on an iOS device.

      • André van Delft
        Posted Jan 7, 2013 at 12:58 PM | Permalink

        I get this PDF page on my iMac, but not on my iPad; neither in Safari nor in iBooks. The page (sequence number 9 in the PDF) shows up empty.

        On the iPad, you may still see the page: click the “David Holland FOI” at the top of Steve’s article, then go to the “Appendix A Data file 155.pdf” – View as HTML; or just click here:
        http://www.whatdotheyknow.com/request/139112/response/343246/attach/html/4/Appendix%20A%20Data%20file%20155.pdf.html

    • Posted Jan 7, 2013 at 1:51 PM | Permalink

      It bears repeating:

      The larger issue is why Sommer wasn’t given an opportunity to do a proper analysis

      • ianl8888
        Posted Jan 8, 2013 at 2:30 AM | Permalink

        Worse, he seems to have been actively prevented from doing it

    • Posted Jan 8, 2013 at 1:29 AM | Permalink

      Well – for 9,000 pounds (which I believe was Sommer’s fee) an IT person, competent or otherwise, would have turned the archives from whatever mail program into plain text files that could be searched using standard textual analysis commands, and completed this in a few hours.

      Sommer appears to have invested the UEAs money in a discussion of the relative merits of different email clients!

      I am not sure how many hours of ‘Sommer time’ 9,000 pounds buys. It also is not clear why a short timeline was imposed, when the report was not completed for around an additional two months.

      But doesn’t it appears that Sommer was either incompetent or was paid to play along with the constraints and obfuscations rather than to actually find anything out?

      • ianl8888
        Posted Jan 8, 2013 at 2:44 AM | Permalink

        1) 5434.38 pounds including VAT@17.5% – see the copy of the invoice

        2) I don’t want to keep repeating this, so last time: he did not receive the thumbdrive copies of the emails until Thursday May 13 pm; he was restricted to working in a London police office between 9am and 5pm, no weekends; he could not remove the thumbdrives to his own office overnight; he was required to produce a preliminary (draft) report by Monday May 17 pm, thus giving him 2 days to analyse and report on 8Gb of email data in a Eudora format

        3) my tentative conclusion is that he was never meant to succeed, just another part of the whitewash

        • Posted Jan 8, 2013 at 10:42 AM | Permalink

          Yes, Sommer was clearly not intended to succeed. I remembered the 9,000 pounds figure from the David Holland comment here: http://www.bishop-hill.net/blog/2012/1/17/more-from-norfolk-police.html

          (Holland also notes that given the climategate emails, he was able to find relevant information in minutes, despite not claiming to be an ‘IT Expert’.)

        • Posted Jan 8, 2013 at 12:58 PM | Permalink

          Ah, it takes a real IT Expert to not find information this obvious!

    • Posted Jan 8, 2013 at 11:58 AM | Permalink

      Re: ZT (Jan 7 12:21), Welllll if we are going to consider “larger issues”… here are a couple — at least…

      Let’s speculate about whether the person who contracted the “auditor” knew what was involved and what the hurdles would be — technically, politically and legally. If they were aware of the hurdles — that leads to one set of conclusions. If they were not aware of the hurdles — that of course leads to a completely different set of conclusions — in all likelyhood.

      On a higher plane — what about the person directing the auditor’s boss? Were they aware that they had a chain of people that were simply unaware of how to do an audit/investigation of this mature — referring to the entire investigation of course.

      The point is that you can go up the entire chain with that particular mode of speculation — wondering about the competence — or lack thereof — of the people directing the investigative efforts of those below. Was everybody put in place because of their skill — or the lack of skill?

      Rather Pythonesque or “Ministerial” was it not? Perhaps it was worthy of Anthony Eden’s best efforts — or was it just serendipity that we had a chain of people that simple could not investigate much of anything? If that seems unlikely it does lead to the most grotesque of speculations — does it not? 😉

  4. DGH
    Posted Jan 7, 2013 at 12:39 PM | Permalink

    There are many downloads available from pmsommer.com.

    In his CV he referes to himself as Professor Sommers and there are a couple of papers/conferences in which he was a participant that refer to him as Professor. In all of the papers and powerpoint presentations that that I read he refers to himself without the title.

    One of the papers has a digitial signature line. http://www.pmsommer.com/DI_hackingtools.pdf The signature is right justified.

  5. David Jay
    Posted Jan 7, 2013 at 12:51 PM | Permalink

    As to formatting, I often run into the MS Word formatting preferences when I cut and paste.

    MS Word allows you to paste using the format of the original document or of the new document. If the editor’s profile called for using the destination document formatting, copying in the original document into a new document would reformat the text to the editor’s default preferences.

  6. David Jay
    Posted Jan 7, 2013 at 12:53 PM | Permalink

    make that: “copying in the original TEXT into a new document”

  7. Posted Jan 7, 2013 at 3:43 PM | Permalink

    The erratically appearing page 3 seems (on my Windows 7 laptop) to have a slightly different format to the rest of the document – it is in greyscale while the rest are in black and white. It also has the only redaction in the document, which may explain the format change: that page was edited with image processing software?

    • Steve McIntyre
      Posted Jan 7, 2013 at 3:48 PM | Permalink

      Jonathan, what do you think of the possibility that the short document was prepared by someone other than Sommer? And if so, does the form of presentation seem odd to you?

      • Posted Jan 8, 2013 at 3:37 AM | Permalink

        Re: Steve McIntyre (Jan 7 15:48), It’s hard to say much with certainty. The long document reads completely authentic to me, and the form is dictated by the circumstances: he is claiming for one week’s work (the “standard working week” in much of the UK public sector is 37.5 hours) with essentially nothing to show for it, so he documents everything in great detail to show UEA that it is not his fault. The short document shows his turn of phrase, but the layout has a faint whiff of academic bureaucrat.

        My suspicion would be that either (i) he was asked for a shorter less detailed statement suitable for transmission to the enquiry, or (ii) somebody at UEA prepared the shosrt statement as a precis of the long document and then asked him to approve it, but it’s difficult to say which of these is more likely to be true. The doucument properties are interesting, but simply show that somebody else typed it up, or perhaps reformatted it from text sent by email, so the wording could still be entirely authentic.

      • Don McIlvin
        Posted Jan 11, 2013 at 8:31 PM | Permalink

        The 2 pager struck me as having an element of being agenda driven. Where as Sommer’s long document is a compilation of explicit technical detail as to how to proceed with no advocacy.

        An example of this is his forecast for a next steps approach requiring first 1-2 days for one task then as much as an additional 2 days for another where there after searches by the key words provided could commence. In this sense he is providing factual justification for the time. There is certainly a prospect that he would proceed, pending a decision to do so.

        In the 2 pager, this time required is conflated to requiring “weeks”, and presented as supporting a notion that the time involved would be unfeasible.

        The 10 pager articulates technical steps to proceed involving 3 to 4 days, with some uncertainty but also immediate opportunity for results there after.

        The 2 page “summary” advocates weeks of work to get results, implied as unfeasible with out any further summary of technical activity. It is also obvious this document advocates a decision not to proceed. Sommer gets paid if they proceed, so advocating vague conflated time frames is a cross purposes to his 10 page document.

        One is about technical methodology and a time projection. The other is all about a contrived reasoning for not proceeding.

        Two completely different purposes and mindset. It is not a summary.

        I like the idea he didn’t write it but approved it nominally. But I would add he having already been told the jig was up, and that this likely occurred in an wrap-up meeting where deviation in details no longer mattered to him.

  8. NiV
    Posted Jan 7, 2013 at 4:17 PM | Permalink

    For those interested, definitions of UK classifications can be found by Googling “security vetting and protective markings: A guide for emergency responders” and looking for paragraph 3.25. In my view, it’s not secret, unless of course there’s something in them we don’t know about.

    They probably suspected it was the sort of thing that people might want to leak, and set the level so it didn’t.

  9. James Smyth
    Posted Jan 7, 2013 at 5:36 PM | Permalink

    It is odd that the form has changed in such a way that the language has been cleaned up and the detail has changed or disappeared. Not only have specific names (Briffa) been changed to researcher “A”, but much of the specific detail, including varying email client apps, is gone.

    That was just at a glance.

    • James Smyth
      Posted Jan 7, 2013 at 5:43 PM | Permalink

      Why would even a polished “initial report” obscure details of changing email tools (database and file formats). Note how section 18 (“i am unable to say of my own knowledge that the material is complete”) is gone from the shorter version.

  10. matthu
    Posted Jan 7, 2013 at 5:55 PM | Permalink

    Paragraph 18 seems quite interesting including the following:

    “… I am not examining the material in the formats in which they were originally located on the various UEA machines. Qinetiq have provided the material in a “convenient” form. Although I have no reason to doubt their expertise I am not able to say of my own knowledge that the material is complete and reliable in its own terms. This “convenient” approach also limts the extent to which I comment on how the published emails were selected.”

    [ Interesting choice of the word “convenient” ]

    Then in paragraph 26

    “Because I did not complete indexing I have been so far unable to use the keywords with which I had been provided … ”

    and paragraph 27

    “More sophisticated searching is possible …”

    and paragraph 33

    “Because of time constraint I have not at this point looked at the computer m-m-crupdj5…”

    and paragraph 34

    “… I currently have no idea about the level of hits on the keywords.”

    [ It seems obvious why the inquiry might want many of these observations to be suppressed from public knowledge. ]

  11. Jean S
    Posted Jan 7, 2013 at 6:40 PM | Permalink

    As a non-native English speaker, the 2 page document seems somehow “better”/”cleaner” English to me than the 10 page document. So I think either Sommer prepared the 10 page document in haste and put more effort (later) on the 2 page document, or the latter is written by someone else.

    A thing that caught my eye is the use of quotation marks. To me it appears that that the use is slightly different in two documents, but it is hard to me to say if a writer could produce both cases given, for instance, the other document is written in haste and more effert is put on the other. Sommer seems to be using either single quotes (or no quotes) for program names, e.g., `Thunderbird’, ‘Eurora’, but in the 2 page document “Thunderbird” is written with double quotes (twice). Also Sommer seems to be a rather heavy user of single quotes (for all file and computer names) but the 2 page document does not contain a single example. Notice also: “thumb drive” vs thumbdrive

    However, I think the most important finding is that in the first paragraph of the 2-page document:

    They were believed to have been “hacked” from a CRU back-up server which itself was
    removed by the Norfolk Police as part of their investigation into the “hack”.

    Not only the use of the word hack in quotation marks is different (unlike in the Summer’s letter) from the 10-page document (Climategate Hacked Emails scandal), but more importantly, the fact that emails were coming from a single back-up server is infromation that is not contained anyhere in the other document. Instead, Summers just says they were “hacked from the [sic] server”, and Sommer clearly states that he does not know how the emails were obtained (see 46–48). So I think it can be safely concluded that 2 page document was either written by Sommer on a later day (after receiving extra information), or it was written by someone else.

    • Steve McIntyre
      Posted Jan 7, 2013 at 7:31 PM | Permalink

      The quotation marks are interesting.

      An obvious point that I should have mentioned before: in their FOI production of correspondence with SOmmer, the UEA did not produce the two-page version later published by the Muir Russell panel. So either they withheld some SOmmer correspondence (without disclosing the withholding) or SOmmer didn’t send them the two-page report. Reasonable people can favor either. My own working surmise right now is the one indicated by the Document Properties: that the two-page version was in fact prepared by William Hastie. Like almost everything to do with Muir Russell, it’s all very odd.

      • Posted Jan 8, 2013 at 5:12 AM | Permalink

        The FOIA does not require a public authority to produce any document that is already publicly available, so it is not at all surprising that UEA did not provide David Holland with a copy of the document published by Muir Russell.

        Steve: nut it requires the UEA to refuse on the basis that the document is available elsewhere. BTW I have an FOI which asked the UEA to produce any documents provided to Muir Russell that had not been archived. If they supplied Muir Russeell with the long version, they should have disclosed it.

        • HaroldW
          Posted Jan 8, 2013 at 7:58 AM | Permalink

          Tony: “is not at all surprising that UEA did not provide David Holland with a copy of the document published by Muir Russell.”
          But there would likely be a cover letter, and in any case I’d expect the FoIA release to mention the omission, citing the appropriate section of the regulations.

          Steve – that would be the “Hastie pudding hypothesis”, then?

      • Posted Jan 8, 2013 at 4:23 PM | Permalink

        Steve and Harold

        It would be nice to think that the UK FOIA works in the way you both seem to envisage. It does not. FOI officers in public authorities know that they have nothing to fear from the ICO or Information Tribunal if they commit minor breaches of procedure.

        It could be argued that UEA’s letters to David Holland (see link in header post) satisfy FOIA Sec 1 (1) (a) [the requirement to confirm that information is held] as his request was not for a specific document such as an attachment to an identifiable email. What they told him about the way in which they were processing his request implied that they held information, even if this is not stated explicitly.

        So far as FOIA Sec 1 (1) (b) is concerned [the requirement to release information] see Sec 2 (2) (a) and Sec 2 (3) (a) disposes of that requirement. The two-page report was subject to an absolute exemption. No public authority is going to provide a manifest of all the documents that might possibly fall within the scope of a fairly broadly expressed request like this one together with reasons for not releasing some of them. They know that nobody is going to require them to do so, particularly where documents are easily available to the complainant other than by invoking the Act.

        In an astonishing case I brought against DECC earlier this summer, which hasn’t reached the media yet, neither the ICO nor the IT could be persuaded to recognise a far more blatant disregard for the requirements of the Act than those you are considering, let alone censure the public authority. If I have learned anything from the two hearings I was involved in, it is that there is a wide gulf between what the Act says and the way in which it is applied. The ICO and public authorities understand this very well.

        I’m sorry to be didactic, but there is a world of difference between reading the Act, as the litigant in person is likely to do, and understanding how it will be applied in court. FOIA officers and their lawyers deal with this stuff every day and in my experience they know just how to work the system most of the time.

        • HaroldW
          Posted Jan 8, 2013 at 8:27 PM | Permalink

          TonyN –
          Thanks for the correction. I didn’t realize you were *that* TonyN. I defer to your direct experience vs. my naive reading of the rules.

      • Posted Jan 9, 2013 at 5:41 PM | Permalink

        It’s been a big eye-opener and maybe the scars are still a bittender. I hope SteveM is prepared.

    • sue
      Posted Jan 7, 2013 at 8:48 PM | Permalink

      I think the single quotes are from Qinetiq. Perhaps a copy and paste. Notice how paragraphs 16, 28 and 35 begin with Qinetiq says: ….

      Paragraph 30 (last 2 sentences): Jones had a Denier mbx 🙂

    • Wayne2
      Posted Jan 7, 2013 at 9:44 PM | Permalink

      It may just be me as an old-school programmer, but I’d never use double-quotes for variable names, file names, or program names. If I felt the need to use quotes, I’d use single-quotes. I wouldn’t use any quotes for program names which are already capitalized, such as Eudora.

      Double-quotes would be used where I’m quoting someone, expressing skepticism, or implying that a phrase is not technically correct but is for a non-technical audience.

      If Sommer wrote the 10-page version — which does sound more techie — and then the 2-page version, I’d assume that he was either instructed to use punctuation like a non-computer-geek, or someone else did a quick substitution to “clean up” the non-standard punctuation. I think the evidence that someone else summarized/edited the 10-page into a 2-page is pretty compelling.

  12. Lance Wallace
    Posted Jan 7, 2013 at 7:28 PM | Permalink

    A little British understatement here…

    After pointing out that the original emails were in Eurdora and the Norfolk plod’s consultant Qinetiq converted them to the less-useful Thunderbird, Sommer comments in paragraph 46:

    (In fact Eudora’s facilities are rather more sophisticated than those in Thunderbird, and allows searching against regular expressions. This rather raises the question why Qinetiq chose to convert the mailboxes to Thunderbird).

    Could the reason be to further muddy the waters?

    • Posted Jan 7, 2013 at 7:56 PM | Permalink

      You mean they’re still waters? I would say it was to further muddy the mud.

      Good understatement though.

  13. drissel
    Posted Jan 7, 2013 at 7:33 PM | Permalink

    Do British writers prefer single quotes where Americans would use double qutoes?

    • Graeme W
      Posted Jan 7, 2013 at 8:40 PM | Permalink

      As I understand it, the literary profession in the UK uses single quotes for dialogue, while the US uses double quotes. I don’t know if there’s a standard for using quotation marks for non-dialogue, though I suspect it follows the dialogue rule.

      You can see this in any published book, based on where it was published. Many of my novels published in the UK have single quotes for character dialogue, while the books published the in the US have double quotes.

      • Watchman
        Posted Jan 8, 2013 at 8:48 AM | Permalink

        Unusually something I can contribute – the use of quotes is very variable in humanities, often dependent on the publisher (if they are concerned). There are ‘conventions’, but they are not exactly universly approved.

        What is normal is that people drafting pre choice of publisher/producing their own reports tend to adopt their own style consistently. Inconsistent usage is one of those things that often indicates plagiarism in student essays as well.

  14. Posted Jan 7, 2013 at 9:10 PM | Permalink

    They were worried about being burned:
    The first thing they hoped Sommer learned
    Was “how much was still out?”
    They had reason to doubt
    And feared even more. That fear was earned.

    The size Sommer computed for mails
    Dwarfs releases — the six thousand pales:
    CG2’s likely stored
    The complete whored horde’s hoard
    We will see more of massive Team fails

    ===|==============/ Keith DeHavelle

    • theduke
      Posted Jan 7, 2013 at 10:18 PM | Permalink

      “. . . The complete whored horde’s hoard . . .”

      Now that’s what I call “wordplay.” Well done, Keith.

  15. ianl8888
    Posted Jan 7, 2013 at 11:28 PM | Permalink

    On the 10 page report …

    I may well be missing something here, but Sommer summed up one of the aims of his work as attempting to deduce search strings or the like as may have been used to pick out emails that were released by FOI2009, in the hope of identifying further related emails that may now reside in the encrypted ZIP file from FOI2009

    Sommer states, several times and quite emphatically, that because the Norfolk Constabulary withheld the thumbdrives from him on security grounds until very late in his allotted time period, he had no opportunity to index the emails and then institute analyses looking for ways to answer this aim

    So exactly what use was Sommer’s report to anyone ? (This is no reflection on Sommer’s competence or integrity, rather it seems to me that he was deliberately set up to fail … and I think he recognised that quite early on)

  16. Posted Jan 8, 2013 at 12:21 AM | Permalink

    The “Two Document Debate” is far less interesting than going through the longer report and finding that “nothing” was discovered regarding what was in the email and that apparently no searches were made and emails were found of any relevance to “what???”.

    So he charged about 5500 Pounds to report that he had nothing to report? To me that is astonishing.

    I have done similar work on packages of files of similar size with Eudora and Exchange and Thunderbird and Groupwise and others and moved the data between them for various reasons…. sometimes with many thousands of emails and attachments. I do not recall having the speed problems that he describes but perhaps he was working with an “antique” computer — than can happen at “secure” establishments if they are loathe to change the equipment.

    Simply put, I do not understand why he had such difficulty — one day should have revealed much — not enough — but still… Thunderbird did indeed have sophisticated filters in 2010 — they could have sorted out some of the email and placed it in folders based on sender, recipient, key phrases etc. Maybe accessing those filters is not as I obvious as I thought — but I used them for automatic sorting of mail and segregation and sorting boxes with several thousand emails… I do not recall the long delays — maybe 15min — 1/2 hour on very large collections — but still — not that bad.

    I still use Thunderbird for some accounts — though we have our own server for the corporate stuff now — but, if I want to, I can make Thunderbird an IMAP client to our server and it will upload the accounts to a “database” style server — in about an hour and do a much faster sort. Knowing I might need that capability, if the job was that important I might have gone with the server loaded then tied in the Thunderbird client to let the server do the heavy lifting. This is grunt work — not even sophisticated investigation…

    Since he was permitted to download other software I presume that he could have downloaded some of the mail server packages in demo form and installed and tested them — that would have used two hours — but returned the investment many fold.

    Plus if I recall correctly they were simply text files and I did indeed write standalone scanners from time to time if the need was great.

    Bottom line — I don’t get it. Did they have the wrong guy for the job? This whole thing has an unbelievable air to me. Maybe it wasn’t his cup of tea — and he did not wish to say so.(???) I can only speculate.

    That to me these are far more interesting questions.

    Steve: If the expurgated version was produced by someone other than Sommer and backdated, that would be more interesting to me than whether Sommer did as much as he could in a day or so. If SOmmer’s report was done on May 17 and he only had access to the files for a short time on May 14 and on May 17, it’s also hard to see where the 37 hours come from.

    • Steve McIntyre
      Posted Jan 8, 2013 at 1:13 AM | Permalink

      There’s another puzzle in the timeline. Sommer didn’t get the files until late on Friday May 14.

      However, Qinetiq was given the go-ahead to extract the files on April 22 on the condition that they be finished by April 26. On April 22, Qinetiq gave a progress report:

      We have identified seven machines belonging to the individuals concerned and also the significant number of backups of each of these machines. At present I am not able to tell you how many emails will be recovered but will update you as soon as I know. We are currently in the process of recovering the emails for Prof Keith Briffa, and hopefully as long as there are no technical issues we will attempt to get this completed by Monday. [Apr 26]

      There is an email from Qinetiq with date redacted saying to Guy of Norfolk that Briffa and Jones extracted and waiting go-ahead to send to Sommer.

      However, as of May 12, nothing had been sent to Sommer. An email of May 12 says that they had authorized DC Steve Mitchell at Wymondham (Norfolk) to receive data and then courier to London for examination by the “appointed barrister”. In the afternoon of May 14, the files arrived to SOmmer.

      SO why did it take 18 days from April 26 to May 14 to get the files from Norfolk to SOmmer. And why would there be a need for examination by an “appointed barrister”; I wonder who that was.

      • Posted Jan 8, 2013 at 1:44 AM | Permalink

        Sommer read law at Oxford, according to his CV (and ‘spent 13 years as a book publisher’ – which seems rather a severe term). But one gets the general impression from his web site that Sommer is more a ‘professional witness’ than an IT expert. Perhaps, in the standard climatological qualification inflation fashion, a law degree and writing background became IT-expert barrister?

        • Posted Jan 8, 2013 at 9:54 AM | Permalink

          Re: ZT (Jan 8 01:44), That of course was the point I was trying to make without saying exactly so — that maybe, just maybe, he did not have the expertise for the problem at hand. Based on perusing his web site I had similar thoughts as you — that his expertise actually lay elsewhere. Mostly though it was based on the way he described the problem in his long write-up. The web site tour was to confirm that suspicion — that he was not really an email/db expert. Impressions can be misleading though — so at this point I can only guess — though I would make a bet on it and not stint the cash.

          First conclusion — he is not using the language I would use — or have seen used by other people writing a report about reviewing/auditing a set of email mailboxes looking for evidence of something or a pattern.

          It seemed to me that he was completely lost and did not know how to achieve anything in the time allotted. So, if I saw his report — I would want it re-written to hide the fact that someone who did the analysis would not likely have found much of anything about anything in any case. I am not sure that giving him a couple of weeks to do a three day or two day job would have improved the situation.

          Second conclusion: The report had to be re-written so that the fumbling was not so obvious.

          Did he write the second report? Not if he was convinced he was doing his job the first time around. That assessment would require that you know the person and could judge his actual competence for the subject matter at hand.

          Have I written or re-written such reports to cover up such a fumble? I would not want to say. Confidentiality and all that — so no comment. But really, the second report is just gathering a few phrases of the original report to emphasize the lack of time and the reason for nothing to say — while hiding the reason for nothing to say — i.e. the inability to find anything to say — for whatever reason. SO was he set up to have thew inappropriate equipment? Did he bring his own inappropriate equipment, did he incorrectly analyze the problem to be solved, did he lack the skills to perform the work i a reasonable time? Or, perhaps, it was all of the above (my guess).

          But, yes, the report should have been rewritten.

          I leave you with those additional thoughts.

      • ianl8888
        Posted Jan 8, 2013 at 6:07 PM | Permalink

        SO why did it take 18 days from April 26 to May 14 to get the files from Norfolk to SOmmer

        Perhaps because at that point, Norfolk was not at all convinced that someone from UAE was not involved in the original imbroglio ? Sommer was hired by UAE for this job

    • Watchman
      Posted Jan 8, 2013 at 8:54 AM | Permalink

      Would the 37 hours include time spent establishing credentials with police etc as mentioned in the letters – and presumably chasing up where the reports were. I’ve done similiar contract work (non-legal unfortunately) and would bill for preparatory work. Professor Sommers has a single rate cited, so preparatory work is not cheaper (a long phone call was therefore expensive…).

  17. martinbrumby
    Posted Jan 8, 2013 at 5:23 AM | Permalink

    Fascinating, although it will only ever be a ‘footnote in history’.

    But I must point out an obvious point.

    Personally, I think the “cock-up” interpretation of events and actions is usually to be preferred to the “conspiracy” theory.

    But the more I read articles like this and Doug Keenan’s expert filleting of the MET Office over on Bishop Hill
    http://bishophill.squarespace.com/blog/2013/1/7/climate-correspondents.html
    not to mention the whole Hockey Schtick / Climategate / CG Inquiries saga, the harder it is to cling to the idea that the warmists are just chumps and bunglers.

    Chumps and bunglers they flagrantly are. But it is impossible to believe that there isn’t a generous seasoning of what can only be described as ‘conspiracy’ mixed in.

    I guess that we have to remind ourselves that hypochondriacs from time to time do fall sick and the paranoid are sometimes picked on.

    No doubt most of this can be explained (if not excused) by Noble Cause Corruption. But it is quite clear that the UEA, senior Civil Servants (notably Beddington), the fuzz, Oxburgh, Russell, the Royal Society leadership – all have had their little heads together at some stage in a deliberate attempt to be ‘economical with the truth’.

  18. Posted Jan 8, 2013 at 6:09 AM | Permalink

    The whole point of UEA buying their own IT forensic report, is that it gives them ownership of the top copy. What redacted version they pass on to anyone else is then within their control.

    Pointman

    • Posted Jan 8, 2013 at 8:05 AM | Permalink

      Yep, buying it, making sure it can’t find anything, then redacting it. Not that they’ve got anything to hide 🙂

      • ianl8888
        Posted Jan 8, 2013 at 6:01 PM | Permalink

        Pretty much where I’ve arrived

        Yet another tile in the whitewash mosaic

    • Skiphil
      Posted Jan 8, 2013 at 9:32 AM | Permalink

      Re: Pointman (Jan 8 06:09),

      UEA pretended to launch a rigorous inquiry but it was all pantomime*

      * a low form of musical comedy particularly aimed at children and bunnies like Eli Rabbet

  19. fastfreddy101
    Posted Jan 8, 2013 at 8:58 AM | Permalink

    “So which story do you prefer?”
    “The one with the tiger. That’s the better story.
    “Thank you. And so it goes with God.”
    (From the Life of Pi)

  20. David Holland
    Posted Jan 8, 2013 at 9:14 AM | Permalink

    For the total cost of the aborted email extraction project – another pet project of Sir John Beddington – do not forget the £8,910 paid via the police to QinetiQ

    Will is Will Hardie, not Hastie. For his services plus the odd meeting room, video and teleconference the UEA paid the Royal Society of Edinburgh some £27,537.16. At the time its General Secretary was one Geoffrey Boulton – by far the biggest single ICCER fee earner, trousering some £52,500.

    Will has a couple of RSE email addresses and used one from cce-review.org to tell me that the ICCER would not be puting my submission on its website. ‘Whois’ reports that Luther Pendragon registered cce-review at the UEA’s address with Lisa Williams as the admin email contact. However for all his correspondence with the UEA and the ICCER team he used a tiscali address.

    RSE have stated:

    The Royal Society of Edinburgh (“the RSE”) will handle any requests it receives for information through rights under the Freedom of Information (Scotland) Act 2002 (“the Act”), which came into force on 1 January 2005, as if it were required to comply with the Act.

    Do we have Scot brave enough ask? It is a different country. If so be sure to add a protect preserve notice.

    • David Holland
      Posted Jan 8, 2013 at 12:47 PM | Permalink

      Re: David Holland (Jan 8 09:14),
      I am reliably informed that a brave Scot had indeed asked the RSE some questions. So please don’t anyone else do it.

  21. Keith Sketchley
    Posted Jan 8, 2013 at 6:46 PM | Permalink

    Are there any facts that weigh much on the side of anything more than a combination of the behaviour of Microsoft Word and the incompetence of individuals in using Word and other computer software?

    Word processing software – especially Microsoft Word – has a habit of unstable documents due to factors such as default printer (which can change margins and other attributes), whose computer is used, and keeps their documents separate unless set otherwise), and tab settings. (A new document should default to identify the User logged in at the moment – Windows can have multiple users defined, besides Administrator. Formatting attributes may be unique to each user. Note for example that style of quotation marks can be changed between simple, handed, and a clever Microsoft version.)

    Most people do not have any idea that a document contains information that is not normally visible. Stories abound of contract revisions being revealed while showing a couple of document assemblers how to check and change the hidden author attribute Stephen showed herein I found that a proposal to supply equipment to the program I was working on had been prepared earlier for a competitor. This was a military program, fortunately for confidentially the competitor’s name was no surprise to us, I did not explore the document further. The supplier was a large company with long experience in the field including in military work, but of course bureaucracies can’t do the simplest things right. (Word used to default to fast save, which kept revisions until SaveAs was used, IIRC at SP 3 to Word 2003 the default was changed to normal save.)

    Qinetiq’s change of email format could have been laziness in preparing to read the messages. Actually that’s incompetence since proper police procedure is to handle documents very carefully to be robust as evidence.

    While I and many others would not be surprised at more dishonesty, I don’t see evidence for it in this narrow subject. Perhaps the anomalies are reason to ask for more, if worth the bother. They are reason to be wary at least. But no basis for a conspiracy theory.

  22. André van Delft
    Posted Jan 9, 2013 at 9:10 AM | Permalink

    Steve wrote: “Steve: Sommer’s business requires him to maintain confidentiality with customers. There is zero chance that he will respond to questions. It’s not even worth trying”.

    That made me send the following mail yesterday:

    [Begin Quote]
    Dear Mr. Sommer,

    Yesterday I read the blog post “A New Puzzle: Two Versions of the Sommer Report” by Steve McIntyre on ClimateAudit.org. This is about the report you have made for the UEA-CRU review. It has become clear that two versions of your report exist: a 10-page version and a 2-page version. It seems that you created the 10-page version; it is not clear whether you also created the other one.

    I intend to post a comment on that blog post, and for that purpose I would like to receive your answer to the following questions:

    1. Can you explain the fact that there are two versions of the report?

    2. Did you create both versions of the report?
    If so, what were the creation dates?
    If not:
    2.1 Which one did you create?
    2.2 About the one that you did not create: did you cooperate in its creation? Were you aware of its creation? Do you know when it was created?
    2.3 Do you endorse the contents of that version; if not: why?
    2.3 Do you endorse you being named as the author of that version?

    3. Jean S wrote:

    Not only the use of the word hack in quotation marks is different (unlike in the Summer’s letter) from the 10-page document (Climategate Hacked Emails scandal), but more importantly, the fact that emails were coming from a single back-up server is infromation that is not contained anyhere in the other document. Instead, Summers just says they were “hacked from the [sic] server”, and Sommer clearly states that he does not know how the emails were obtained (see 46–48). So I think it can be safely concluded that 2 page document was either written by Sommer on a later day (after receiving extra information), or it was written by someone else.

    Do you have any comment on this?

    4. Did the UEA return to David Holland a copy of the full correspondence between EAA and you in connection to the assignment?

    5. Do you have anything else to say in reaction to the blog post on ClimateGate and the current comments?

    I look forward to your reply, which I intend to quote at ClimateGate. If you would rather post your answer directly to the ClimateAudit blog article, that would be very fine.

    Regards,

    André van Delft
    [End Quote]

    I resent my mail today, because I feared my first mail had been marked as spam, because of hyperlinks. I just received a reply from Mr. Sommer. Steve was right:

    [Begin Quote]
    Dear Andre

    Thank you for your emails.

    You will have to forgive me but I won’t be replying in any detail to your questions. From my perspective this was a limited exercise for a short period in the first part part of 2010 to assist with just one of the various committees reviewing events. My work had a somewhat unsatisfactory outcome for the reasons set out in the two reports which have been released under UK FOIA.

    I have had a brief look at the blogging site to which you refer; some of the bloggers seem to understand my position, others appear to determined to find a conspiracy.

    For what it is worth my own view is that those who are concerned about climate change science would do better to contribute directly to the field with their own basic research rather than spending further time on events which have already been subjected to extensive scrutiny by a large number of agencies and individuals.

    You are of course quite free to quote from this letter. But I won’t be making any further remarks.

    regards

    Peter Sommer
    [End Quote]

  23. Keith Sketchley
    Posted Jan 9, 2013 at 6:06 PM | Permalink

    I clarify that I am commenting only on the theory that different people wrote the two versions.

    If I understand correctly there is a major question of why there are two versions, and other questions including completeness of releases and statements.

    (I garbled the paragraph about hidden information. I intended to say that I have heard many stories of contract revisions being revealed, and tell of my experience in showing some colleagues how easy it was to see information – using a real document I had a better example than I expected.)

  24. Keith Sketchley
    Posted Jan 9, 2013 at 6:13 PM | Permalink

    Confidentiality is a very good principle.

    The problem in climate alarmism is that bureaucrats and their consultants are not subject to market forces, they are supported by government force and have quasi-monopolies.

    In the free market, supported by a justice system, irrational thinking and bad ethics eventually leads to financial failure because customers find someone else. (I walk through much of that in http://www.keithsketchley.com/monopol3.htm.)

    The root problem is voters who fail to develop and elect competent honest officials.

    Officials who cannot think clearly and do not are prone to being conned, and prone to believing in centralized control, so end up funding places like CRU and Penn State and appointing bad people to run them.

  25. Duster
    Posted Jan 12, 2013 at 11:07 PM | Permalink

    Word, fonts and printers. These are a poor combination. I work presently in an office that uses Word and has several different brands of printers either on the network or shared from computers that are on the network. One of the more egregious things that Word does to a document is include metadata regarding the target default printer, the fonts (either MS fonts of built-in or installed printer fonts) and other hardware specifics. In formatting a document and then transferring it to another system, some very irritating issues can emerge. Differences in font kerning can effect line length and thus pagination. Nearest similar generic fonts can be substituted for uninstalled fonts for a specific printer. I have seen very rarely changes in alignment – left aligned becoming centered for no good reason for instance. So, while the documents may be different in specific ways that are suggestive, it probably won’t do to rule out Microsoft and incompetence.

  26. Steve McIntyre
    Posted Jan 14, 2013 at 9:55 AM | Permalink

    Some further information on the two versions of the Sommer report has come to my attention. The UEA has described the relationship as follows:

    The University was sent an initial draft of Professor Sommer’s report. The University and the review team both took the view that the report was too long and would benefit from being more focused and concise. It appears that those comments were taken on board by Professor Sommer as his final report, which is publicly available on the internet, was shorter.

    I wonder who commented on behalf of the UEA and the Russell Review. And why the UEA didn’t list these commments as part of the requested correspondence.