Anderson Cooper on Penn State Secrecy

Anderson Cooper had a segment on Penn State secrecy last night, with his contributor Drew Griffin expressing extreme frustration at Penn State’s stonewalling of information that would be routine at other institutions. Several years ago, Penn State lobbied for (and got) an exemption from Pennsylvania FOI legislation. (Given the exemptions in the state FOI legislation, I’m not sure that simply removing Penn State’s exemption would accomplish everything that people think it would, but that’s another story.)

In a panel segment that was on air last night (not in the online clip), one of the contributors described Penn State as “arrogant”, “imperious”.


72 Comments

  1. Posted Nov 16, 2011 at 10:30 AM | Permalink | Reply

    By the time they’ve finished paying lawyers and victims they’ll have sufficient money to fund a little league hockey team.

    snip – you’ve already said this re Spanier on a few occasions. Not relevant to the point.

  2. thisisnotgoodtogo
    Posted Nov 16, 2011 at 12:56 PM | Permalink | Reply

    PR firm must have googling under control. Nothing about this development if you google “Penn State.

  3. Matt Skaggs
    Posted Nov 16, 2011 at 1:27 PM | Permalink | Reply

    When PSU got its exemption from disclosure laws in 2007, here is what Spanier said (from CNN):

    “Nobody would argue the point that the public has a right to know how public funds are spent,” Spanier said at the time. “But these proposals will fundamentally change the way we operate, the way our trustees govern and the way the university administers their policies.”

    In hindsight, that sure looks like a red flag.

    • stan
      Posted Nov 16, 2011 at 5:43 PM | Permalink | Reply

      “Nobody would argue the point that the public has a right to know how public funds are spent,” Spanier said at the time. “But these proposals will fundamentally change the way we operate, the way our trustees govern and the way the university administers their policies.”

      Let’s hope so! The way he and Penn St operated certainly belongs high on any list of things that need to fundamentally change.

  4. P. Solar
    Posted Nov 17, 2011 at 4:59 AM | Permalink | Reply

    Steve , sorry for the OT. I have a climate auditing question.

    I have found what looks like clear proof of Jones’ “cooling buckets” adjustment being grossly over done and probably 5y too late.

    Looking at second time differential is revealing because ….- snip — in dT/dt and d2T/dt2

    http://oi44.tinypic.com/2mxr0x4.jpg

    It seems Jones’ adjustment was neither. It also flags B-est treatment of circa 1990 as anomalous

    The fact the these records do generally agree apart from these significant deviations is encouraging, though not entirely unexpected since they are largely using the same data.

    I would like to do the same analysis on “pre-bucket” hadSST2. Do you have such a copy of hadSST monthly?

    Best regards.

    Steve- interesting graphic. All hadSST is post-bucket adjustment. I’ve down estimates in the past of the amount but they are just estimates. You should place your code and results online so that people can look at your calculation easily.

    • James Smyth
      Posted Nov 17, 2011 at 4:53 PM | Permalink | Reply

      Looking at second time differential is revealing because ….- snip — in dT/dt and d2T/dt2

      http://oi44.tinypic.com/2mxr0x4.jpg

      It seems Jones’ adjustment was neither. It also flags B-est treatment of circa 1990 as anomalous

      What was snipped and what does “neither” reply to?

      Should I know exactly why a second derivative w/ an obvious cycle is interesting?

      • P. Solar
        Posted Nov 17, 2011 at 5:32 PM | Permalink | Reply

        The censors scalpel did make the post a little confusing (though it was primarily just a data request to Steve).

        The point is the 1945-50 section where the blue line ( had SST2) deviates grossly from tendencies in the other records.

        It is hard to see how SST could be showing very rapid acceleration without the land temperatures reflecting this in some way. This period is where M.O. Hadley are applying Phil Jones’ adjustment for a difference in temperatures supposedly caused by Americans using buckets or some such.

        This plot shows that adjustment, if justified at all, is being grossly miscalculated.

    • P. Solar
      Posted Nov 17, 2011 at 5:55 PM | Permalink | Reply

      Steve, the calculation is pretty simple. Just the differentials found by subtracting successive points in the monthly data. Done twice to get 2nd diff.

      A 24 month gaussian is applied when plotting to remove enough short term noise to see the cycles.

      I will post a script but I wanted to find older pre-bucket version to clearly show the effect of Jones’ trick rather than just implying it by comparison to other data.

      It’s unfortunate this is not one you have had cause to download before. I was hoping this was in your archive.

      Another interesting feature is what happens in B-est around 1990. This is what results in them loosing half the magnitude of the 1998 El Nino. This indicates that something is wrong with their ‘scalpel’ technique.
      Rather than it having cunningly got right what other records got wrong.

      I call this the 1998 mastectomy. (Careful with that scalpel doctor!)

      Like you, I wanted to run their code but the size of the arrays seems to require Fortran libs with 64 bit indices. Hopefully they will provide a more manageable format soon.

      • Geoff Sherrington
        Posted Nov 17, 2011 at 10:49 PM | Permalink | Reply

        Re 1990 feature – it is possible that around the time of a WMO meeting, some countries changed the method of calculating Tmean. Often, before 1990, it was half the sum of Tmax and Tmin obtained from a daily reading of a max-min glass thermometer. After this date, there were more instances of (for example) taking the daily averages of a number of readings each day, from thermistor or thermocouple devices. See, e.g. http://www.bom.gov.au/amm/docs/2004/trewin.pdf

        • P. Solar
          Posted Nov 18, 2011 at 2:35 AM | Permalink

          Yes, having briefly looked at the code it does not seem to be doing anything to deal with T-obs changes that directly relates to the different method.

          It seems that they just use their ‘scalpel’ To split individual records when they see a discontinuity (or where metadata suggests a change).

          While this removes the step change from the record, the ensemble looses one cooler/hotter record and gains a hotter/cooler one at the time of the snip.

          I don’t think this can correctly deal with T-obs changes.

          Clearly B-est has a problem if it is truncating the 1998 event that is so clear in all other records, be they terrestrial measurement or remote sensing.

          I think looking at second differential is useful in seeing where the method is behaving anomalously.

      • P. Solar
        Posted Nov 19, 2011 at 2:17 AM | Permalink | Reply

        OK, that almost worked, at least it doesn’t seem to have made a mess of it. Here’s the gaussian filter;

        #!/bin/awk -f

        # pass input through 3 sigma gaussian filter where sigma, if not given, is 2 da$
        # usage : ./gauss.awk filename
        # optional scale_factor simply scales the output
        # use OFMT=”%6.4f”
        # sigma can be compared to the period of the -3dB point of the filter
        # result is centred, ie not shift. dataset shortened by half window each end
        # check whether data is continuous !!

        BEGIN { OFMT=”%6.4f”
        # ARGV[1]=filename; argv[0] is script name, hence ARGC>=1
        pi= 3.14159265359811668006

        if ( ARGC >3 ) {scaleby=ARGV[3];ARGV[3]=””} else {scaleby=1};
        if ( ARGC >2 ) {sigma=ARGV[2];ARGV[2]=””} else {sigma=2};

        print “filtering “ARGV[1]” with gaussian of sigma= “,sigma
        root2pi_sigma=sqrt(2*pi)*sigma;
        two_sig_sqr=2.0*sigma*sigma;

        gw=3*sigma-1; # gauss is approx zero at 3 sigma, use 3 sig window
        # eg. window=2*gw-1 – 5 pts for sigma=1; 11pts for sig=2; 3 sig=17

        # calculate normalised gaussian coeffs
        for (tot_wt=j=0;j<=gw;j++) {tot_wt+=gwt[-j]=gwt[j]=exp(-j*j/two_sig_sqr)/ roo$
        tot_wt=2*tot_wt-gwt[0];
        tot_wt/=scaleby;
        for (j=-gw;jgsfile;
        ln=-1;
        }

        {
        xdata[++ln]=$1;
        ydata[ln]=$2;

        if (ln>2*gw)

        {
        gauss=0
        for (j=-2*gw;j> gsfile;
        }
        else
        {
        # print $1,$2;

        }
        }

        END {
        print “#gausssian window width = “gw+gw+1″,done”
        print “#output file = “gsfile

        }

        [/source code]

        • P. Solar
          Posted Nov 19, 2011 at 2:21 AM | Permalink

          Crap, here we go. The day WordPress allows a preview….

          Ignore that attempt at posting gaussian. diff.awk looks OK.

          (Steve, please snip that last one got mangled)

    • P. Solar
      Posted Nov 19, 2011 at 2:12 AM | Permalink | Reply

      OK , let’s if I can get WP to play nice today. Here’s the diff script:

      #!/bin/awk -f

      # calculate two-point numerical differential from points ‘step’ apart
      # use OFMT = “%6.4f”
      # [usage] ./diff.awk filename
      # default step=1
      # result is logged at the mid point of the x interval, avoiding adding a shift
      # check whether data is continuous !!

      BEGIN { OFMT = “%6.4f”
      # ARGV[1]=fn; argv[0] is script name, hence ARGC>=1
      ln=-1;
      if ( ARGC >2 ) {step=ARGV[2];ARGV[2]=””} else {step=1};
      print “#diffing “ARGV[1]” with step = “,step

      split(ARGV[1],fn,”.”);
      basename=fn[1]
      diff_file=basename”-diff”step”.dat”;

      print “# “,diff_file >diff_file;
      }

      ($0 !~ /^#/)&&($0 != “”){

      xdata[++ln]=$1;
      ydata[ln]=$2;

      if (ln>step)
      {
      xdiff=(xdata[ln]-xdata[ln-step]);
      ydiff=(ydata[ln]-ydata[ln-step])/step;
      print NR,xdata[ln]+xdiff/2,xdiff,ydiff/xdiff
      print xdata[ln]-xdiff/2,ydiff/xdiff >> diff_file;
      }
      else
      {

      }
      }

      END {
      print “#diff width = “step”, last xdiff=”xdiff” , done.”
      print “#output file = “diff_file
      }

      [/source code]

      • P. Solar
        Posted Nov 19, 2011 at 7:25 AM | Permalink | Reply

        Steve, this one can go. I reposted with the correct formatting below.

        Sorry for need to intervene.
        The main bash script it held up in moderation. Hopefully that will also format correctly.

    • P. Solar
      Posted Nov 19, 2011 at 2:26 AM | Permalink | Reply

      hmm, RTFM, let’s try again: gaussian.awk

      
      #!/bin/awk -f
      
      # pass input through 3 sigma gaussian filter where sigma, if not given, is 2 da$
      # usage : ./gauss.awk filename <sigma=2> <scale_factor=1>
      # optional scale_factor simply scales the output
      # use  OFMT="%6.4f"
      # sigma can be compared to the period of the -3dB point of the filter
      # result is centred, ie not shift. dataset shortened by half window each end
      # check whether data is continuous !!
      
      
      BEGIN { OFMT="%6.4f"
      # ARGV[1]=filename; argv[0] is script name, hence ARGC>=1
        pi= 3.14159265359811668006
      
        if  ( ARGC >3 ) {scaleby=ARGV[3];ARGV[3]=""} else {scaleby=1};
        if  ( ARGC >2 ) {sigma=ARGV[2];ARGV[2]=""} else {sigma=2};
      
        print "filtering "ARGV[1]" with gaussian of sigma= ",sigma
        root2pi_sigma=sqrt(2*pi)*sigma;
        two_sig_sqr=2.0*sigma*sigma;
      
        gw=3*sigma-1;  # gauss is approx zero at 3 sigma, use 3 sig window
      # eg.  window=2*gw-1 -  5 pts for sigma=1; 11pts for sig=2; 3 sig=17
      
      # calculate normalised gaussian coeffs
        for (tot_wt=j=0;j<=gw;j++) {tot_wt+=gwt[-j]=gwt[j]=exp(-j*j/two_sig_sqr)/ roo$
        tot_wt=2*tot_wt-gwt[0];
        tot_wt/=scaleby;
        for (j=-gw;j<=gw;j++) {gwt[j]/=tot_wt};
      
      # strip off last .xxx part of file name
      # improve this  (doesn't work with paths like ../filename)
      
        split(ARGV[1],fn,".");
        basename=fn[1]
        gsfile=basename"-gauss"sigma".dat";
      
        print "# ",gsfile >gsfile;
        ln=-1;
      }
      
      {
        xdata[++ln]=$1;
        ydata[ln]=$2;
      
      
        if (ln>2*gw)
        {
          gauss=0
          for (j=-2*gw;j<=0;j++) {gauss+=ydata[ln+j]*gwt[j+gw]}
          print NR,xdata[ln-gw],gauss
          print xdata[ln-gw],gauss >> gsfile;
        }
      else
        {
      #    print  $1,$2;
      
        }
      }
      
      
      END {
        print "#gausssian window width = "gw+gw+1",done"
        print "#output file = "gsfile
      
      }
      
      
      • P. Solar
        Posted Nov 26, 2011 at 3:54 PM | Permalink | Reply

        someone just pointed out a copy/paste error in gauss.awk as shown above. Line 26 ends in “roo$” and gives an error in awk.

        Here is the non-truncated version of line 26.

        
        # calculate normalised gaussian coeffs
          for (tot_wt=j=0;j<=gw;j++) {tot_wt+=gwt[-j]=gwt[j]=exp(-j*j/two_sig_sqr)/ root2pi_sigma};
        
    • P. Solar
      Posted Nov 19, 2011 at 2:30 AM | Permalink | Reply

      I’ll repost diff.awk since diplaying this properly makes it a lot easier to copy the code accurately.

      
      #!/bin/awk -f
      
      # calculate two-point numerical differential from points 'step' apart
      # use  OFMT = "%6.4f"
      # [usage] ./diff.awk filename <step>
      # default step=1
      # result is logged at the mid point of the x interval, avoiding adding a shift
      # check whether data is continuous !!
      
      
      BEGIN {  OFMT = "%6.4f"
      # ARGV[1]=fn; argv[0] is script name, hence ARGC>=1
        ln=-1;
        if  ( ARGC >2 ) {step=ARGV[2];ARGV[2]=""} else {step=1};
        print "#diffing "ARGV[1]" with step = ",step
      
        split(ARGV[1],fn,".");
        basename=fn[1]
        diff_file=basename"-diff"step".dat";
      
        print "# ",diff_file >diff_file;
      }
      
      ($0 !~ /^#/)&&($0 != ""){
      
        xdata[++ln]=$1;
        ydata[ln]=$2;
      
        if (ln>step)
        {
          xdiff=(xdata[ln]-xdata[ln-step]);
          ydiff=(ydata[ln]-ydata[ln-step])/step;
          print NR,xdata[ln]+xdiff/2,xdiff,ydiff/xdiff
          print xdata[ln]-xdiff/2,ydiff/xdiff >> diff_file;
        }
      else
        {
      
        }
      }
      
      
      END {
        print "#diff width = "step", last xdiff="xdiff" , done."
        print "#output file = "diff_file
      }
      
      
      
    • P. Solar
      Posted Nov 19, 2011 at 6:20 AM | Permalink | Reply

      A simple one to reformat the different monthly formats into one month per line with a floating point date.

      #!/bin/awk -f
      
      # parse monthly data format: " yyyy mm temp " to two colums : f.p. date, temp
      # Crutem3, BEST , etc climate data
      # log f.p. data at mid month ($2-0.5) else dec=12 becomes new year
      # use  OFMT = "%6.4f"
      
      
      BEGIN{ FS=" "; OFMT = "%6.4f"}
      
      ($0 !~ /^#/) && ($0 !~ /^%/) && ($0 != "") && ($0 !~ /^:blank:$/) {
       print $1+($2-0.5)/12.0  ,$3
      }
      
      
      • P. Solar
        Posted Nov 19, 2011 at 7:20 AM | Permalink | Reply

        BTW this one is called : parse-12m.awk

    • P. Solar
      Posted Nov 19, 2011 at 6:52 AM | Permalink | Reply

      This bash script relies on the three previous ones. It should retrieve and process B-est, hadcrut, hadsst2 and crutem3, to give gaussian filtered second diffs.

      Each segment between the breaks can be run separately if prefered.

      It should be turnkey unless there’s some copy paste errors.

      #!/bin/bash

      # ==================================================================
      # bash script for processing SST,CRUT and B-est for d2T/dt2 analysis
      # ==================================================================

      # ==================================
      # Berkeley Earth project

      wget http://berkeleyearth.org/downloads/analysis-data.zip
      unzip -o analysis-data.zip
      # gives Full_Database_Average_summary.txt and Full_Database_Average_complete.txt

      # run parse-12m.awk to give floating point date and temp only
      # and remove % and # comments in header and blank lines
      # truncate file at apparently spurious April 2010 (rem two data lines)

      DATA=’tmp.dat’
      BASE=Full_Database_Average_complete && SRC=”${BASE}.txt”
      ./parse-12m.awk $SRC> “${BASE}.dat” && SRC=”${BASE}.dat” \
      && `wc -l $SRC | awk ‘{print “head -n “, $1-2 , $2 }’` >$DATA \
      && cp $DATA $SRC \
      && ./diff.awk $SRC 1 \
      && ./diff.awk ${BASE}-diff1.dat 1 \
      && ./gauss.awk ${BASE}-diff1-diff1.dat 24

      # ==================================
      # HadCRUT3

      # parse 12 months per line to one month per line : yyyy mm temp
      # and skip lines with supplementary error data
      # run parse-12m.awk to give floating point date and temp only
      # truncate file at end of real data, delete zeros as NaN (rem 3 data lines)

      DATA=’tmp.dat’
      BASE=hadcrut3gl
      wget http://www.cru.uea.ac.uk/cru/data/temperature/$BASE.txt \
      && awk ‘ {i=2; while ((i $BASE.split.txt \
      && ./parse-12m.awk $BASE.split.txt> ${BASE}-monthly.dat \
      && `wc -l hadcrut3gl-monthly.dat | awk ‘{print “head -n “, $1-3 , $2 }’` > $DATA \
      && BASE=hadCRUt3-monthly && SRC=”${BASE}.dat” \
      && cp $DATA $SRC \
      && ./diff.awk $SRC 1 \
      && ./diff.awk ${BASE}-diff1.dat 1 \
      && ./gauss.awk ${BASE}-diff1-diff1.dat 24

      # =========================
      # same for hadSST2

      # parse 12 months per line to one month per line : yyyy mm temp
      # and skip lines with supplementary error data
      # run parse-12m.awk to give floating point date and temp only
      # truncate file at end of real data, delete zeros as NaN (rem 11 data lines)

      DATA=’tmp.dat’
      BASE=hadsst2gl
      wget http://www.cru.uea.ac.uk/cru/data/temperature/$BASE.txt \
      && awk ‘ {i=2; while ((i $BASE.split.txt \
      && ./parse-12m.awk $BASE.split.txt> ${BASE}-monthly.dat \
      && `wc -l ${BASE}-monthly.dat | awk ‘{print “head -n “, $1-11 , $2 }’` > $DATA \
      && BASE=hadSST2-monthly && SRC=”${BASE}.dat” \
      && cp $DATA $SRC \
      && ./diff.awk $SRC 1 \
      && ./diff.awk ${BASE}-diff1.dat 1 \
      && ./gauss.awk ${BASE}-diff1-diff1.dat 24

      # ================
      # CRUtem3

      # parse 12 months per line to one month per line : yyyy mm temp
      # and skip lines with supplementary error data
      # run parse-12m.awk to give floating point date and temp only
      # truncate file at end of real data, delete zeros as NaN (rem 11 data lines)

      DATA=’tmp.dat’
      BASE=crutem3gl
      wget http://www.cru.uea.ac.uk/cru/data/temperature/$BASE.txt \
      && awk ‘ {i=2; while ((i $BASE.split.txt \
      && ./parse-12m.awk $BASE.split.txt> ${BASE}-monthly.dat \
      && `wc -l ${BASE}-monthly.dat | awk ‘{print “head -n “, $1-11 , $2 }’` > $DATA \
      && BASE=CRUtem3-monthly && SRC=”${BASE}.dat” \
      && cp $DATA $SRC \
      && ./diff.awk $SRC 1 \
      && ./diff.awk ${BASE}-diff1.dat 1 \
      && ./gauss.awk ${BASE}-diff1-diff1.dat 24

      [\sourcecode]

      • P. Solar
        Posted Nov 19, 2011 at 8:26 AM | Permalink | Reply

        This one is stuck in moderation. All will make more sense when it goes on line.

        • John M
          Posted Nov 19, 2011 at 10:13 AM | Permalink

          I too have a comment in moderation.

          Happens pretty frequently it seems.

      • P. Solar
        Posted Nov 19, 2011 at 11:19 AM | Permalink | Reply

        OK, let’s try and get the tags right again.

        This version should keep Macists happy, it uses curl instead of wget. Otherwise the same.

        
        #!/bin/bash
        
        # ==================================================================
        # bash script for processing SST,CRUT and B-est for d2T/dt2 analysis
        # ==================================================================
        
        
        # ==================================
        # Berkeley Earth project
        
        #wget http://berkeleyearth.org/downloads/analysis-data.zip
        curl -O http://berkeleyearth.org/downloads/analysis-data.zip
        unzip -o  analysis-data.zip
        # gives Full_Database_Average_summary.txt and Full_Database_Average_complete.txt
        
        # run parse-12m.awk to give floating point date and temp only
        # and remove % and # comments in header and blank lines
        # truncate file at apparently spurious April 2010 (rem two data lines)
        
        DATA='tmp.dat'
        BASE=Full_Database_Average_complete && SRC="${BASE}.txt"
        ./parse-12m.awk  $SRC> "${BASE}.dat"  && SRC="${BASE}.dat" \
        && `wc -l $SRC | awk '{print "head -n ", $1-2 , $2 }'` >$DATA  \
        && cp $DATA $SRC \
        && ./diff.awk  $SRC 1  \
        && ./diff.awk ${BASE}-diff1.dat 1 \
        && ./gauss.awk   ${BASE}-diff1-diff1.dat 24
        
        
        
        
        
        # ==================================
        # HadCRUT3
        
        # parse 12 months per line to one month per line : yyyy mm  temp
        # and skip lines with supplementary error data
        # run parse-12m.awk to give floating point date and temp only
        # truncate file at end of real data, delete zeros as NaN  (rem 3 data lines)
        
        DATA='tmp.dat'
        BASE=hadcrut3gl
        curl -O wget http://www.cru.uea.ac.uk/cru/data/temperature/$BASE.txt  \
        && awk ' {i=2; while ((i<=13)&&($14!="")) { print $1,i-1,$i;i++} }'   $BASE.txt \
                  >  $BASE.split.txt  \
        && ./parse-12m.awk    $BASE.split.txt>  ${BASE}-monthly.dat  \
        && `wc -l hadcrut3gl-monthly.dat | awk '{print "head -n ", $1-3 , $2 }'` > $DATA \
        && BASE=hadCRUt3-monthly  && SRC="${BASE}.dat" \
        && cp $DATA $SRC \
        && ./diff.awk  $SRC 1 \
        && ./diff.awk ${BASE}-diff1.dat 1 \
        && ./gauss.awk   ${BASE}-diff1-diff1.dat 24
        
        
        
        # =========================
        # same for hadSST2
        
        # parse 12 months per line to one month per line : yyyy mm  temp
        # and skip lines with supplementary error data
        # run parse-12m.awk to give floating point date and temp only
        # truncate file at end of real data, delete zeros as NaN  (rem 11 data lines)
        
        DATA='tmp.dat'
        BASE=hadsst2gl
        curl -O wget http://www.cru.uea.ac.uk/cru/data/temperature/$BASE.txt \
        && awk ' {i=2; while ((i<=13)&&($14!="")) { print $1,i-1,$i;i++} }'   $BASE.txt \
                  >  $BASE.split.txt  \
        && ./parse-12m.awk    $BASE.split.txt>  ${BASE}-monthly.dat \
        && `wc -l  ${BASE}-monthly.dat | awk '{print "head -n ", $1-11 , $2 }'` > $DATA \
        && BASE=hadSST2-monthly   && SRC="${BASE}.dat" \
        && cp $DATA $SRC \
        && ./diff.awk  $SRC 1 \
        && ./diff.awk ${BASE}-diff1.dat 1 \
        && ./gauss.awk   ${BASE}-diff1-diff1.dat 24
        
        
        
        # ================
        # CRUtem3
        
        # parse 12 months per line to one month per line : yyyy mm  temp
        # and skip lines with supplementary error data
        # run parse-12m.awk to give floating point date and temp only
        # truncate file at end of real data, delete zeros as NaN  (rem 11 data lines)
        
        DATA='tmp.dat'
        BASE=crutem3gl
        curl -O http://www.cru.uea.ac.uk/cru/data/temperature/$BASE.txt \
        && awk ' {i=2; while ((i<=13)&&($14!="")) { print $1,i-1,$i;i++} }'   $BASE.txt \
                  >  $BASE.split.txt  \
        && ./parse-12m.awk    $BASE.split.txt>  ${BASE}-monthly.dat \
        && `wc -l  ${BASE}-monthly.dat | awk '{print "head -n ", $1-11 , $2 }'` > $DATA \
        && BASE=CRUtem3-monthly   && SRC="${BASE}.dat" \
        && cp $DATA $SRC \
        && ./diff.awk  $SRC 1 \
        && ./diff.awk ${BASE}-diff1.dat 1 \
        && ./gauss.awk   ${BASE}-diff1-diff1.dat 24
        
        
        
        
        
        
    • P. Solar
      Posted Nov 19, 2011 at 7:10 AM | Permalink | Reply

      My graphic was produced with gnuplot . Running the above scripts to process the data and then running this from gnuplot command line should reproduce the graphic.

      Otherwise use your favourite plotting software.

      
      d2SST_g24m="hadSST2-monthly-diff1-diff1-gauss24.dat"
      d2crut3_g24m="CRUtem3-monthly-diff1-diff1-gauss24.dat"
      d2berk_g24m="Full_Database_Average_complete-diff1-diff1-gauss24.dat"
      
      plot s=0 \
      , d2berk_g24m using 1:($2/2.) tit "d2T/dt2 berkeley * 0.5 g24" w l linec rgb "red"\
      , d2crut3_g24m  using 1:($2/2.)  tit "d2T/dt2 Crutem3 * 0.5 g24"  w l linec rgb "green"\
      , d2SST_g24m tit "d2T/dt2 hadSST2 g24"  w l linec rgb "blue"\
      
      
  5. Taphonomic
    Posted Nov 17, 2011 at 2:17 PM | Permalink | Reply

    New allegations of another Penn State coverup of sexual molestation by a proffessor:

    http://abcnews.go.com/US/victim-penn-state-officials-rejected-claims-university-professor/story?id=14960575

    • Steve
      Posted Nov 17, 2011 at 3:03 PM | Permalink | Reply

      Here‘s yet another case involving a Penn State professor who left a tenured appointment for a non-tenured junior appointment. The facts in this case are especially disgusting. The commenter says that the facts strongly suggest a cover-up by Penn State.

      • P. Solar
        Posted Nov 18, 2011 at 2:10 AM | Permalink | Reply

        This get worse the close one looks!

        Perhaps a statistical analysis should be applied here too. Is the number of professors / ex-professors of Penn State involved in serious child abuse representative of the national average of people in similar positions?

        I sincerely hope not, but to be objective we have to ask.

        And if significantly higher, how is it that PSU has developed such a concentration?

      • Posted Nov 18, 2011 at 4:02 AM | Permalink | Reply

        A case disgusting beyond words. Steve, I do salute you for bringing these things to the light.

        This is the detailed argument that Penn State must have been involved in a cover up in the past, after Kyle was convicted by a court in California and sent to prison for over thirty years in May 11. The author is “a professional author and a professor emerita” according to the blog’s About page.

        For Kyle to leave a tenured Associate Professorship at Penn State U. to step down to a non-tenured Assistant Professorship at Cal State East Bay suggests he was fired from Penn State. Tenured faculty get fired only for reasons that are extremely compelling — and those reasons, sadly, have little or nothing to do with their teaching competence or research productivity. Universities get rid of a tenured faculty member only for reasons of grave moral misconduct, such as being falling down drunk or sexual misconduct that makes the university legally liable.

        In Kyle’s case, whatever the misconduct that got him dismissed from his tenured associate professorship at Penn State evidently was hushed up, because he then was hired by Cal State East Bay. Even if his records were sealed at Penn State, Cal State should not have hired him. I’ve sat on countless search committees in my academic career, and someone who left a tenured post for a non-tenured junior professorship would ring so many alarm bells, my head would still be ringing.

        Dan Abrams, the ABC News Legal Analyst, points out that if it can be shown that there is a pattern of the university ignoring complaints of such a serious nature this is classic grounds for a civil case brought by the victims. There seems to be deep trouble for Penn State ahead. …

        • Tim Irwin
          Posted Nov 18, 2011 at 12:57 PM | Permalink

          Richard – I hope you are correct but I don’t think anything will change. Even if the same players were involved, most people will view the Sandusky case and the Mann inquiry as entirely separate issues. Besides, everyone knows the “science is settled.” Any honest attempt to go back and review the inquiry will be met with howls of indignation. Anyone that points out any link will be criticized as equating Mann with a child molester. I have already read comments at other sites that essentially say this already. Steve’s assessment of the culture at Penn State as it relates to the two cases has been twisted into equating Mann and Sandusky. Lazy thinking at best.

        • Posted Nov 18, 2011 at 2:25 PM | Permalink

          I have already read comments at other sites that essentially say this already.

          Me too. But it’s a disgraceful ploy. That’s why I made a point of affirming Steve’s decision to bring these things to the light. As far as I’m concerned it’s deeply relevant as well as tragic.

          I don’t feel like cowering because people might lie about what I’m saying. There’s a simple way to deal with this in one-to-one conversation, which is what I tend to go back to in such situations and that’s to ask: do the latest revelations about Penn State and the way allegations of wrongdoing have been dealt with increase or decrease your confidence in previous inquiries carried out by the leadership there? This has nothing to do with sliming Professor Mann but everything to do with the credibility of the regime within which he’s been working. Seen one way he and many others have become victims of this.

      • Posted Nov 20, 2011 at 6:17 AM | Permalink | Reply

        As I think you mentioned (or intimated) earlier, Steve, if the powers that be at PSU are willing to engage in such willful abrogation of their moral responsibility in such abhorrent matters, their behaviours in the Mann “enquiries” (in hindsight) were almost a foregone conclusion. Even the Spanier inspired(?) excuses are the same.

        Spanier’s successor’s possible allegiances do not give one a high level of confidence that he is a new broom who will sweep clean. Consider: Newly appointed President Erickson’s (undated) service as “head of the Department of Geography in the College of Earth and Mineral Sciences” and his (also undated) service as “chair of the Academic Leadership Council, which provides advice and counsel to the president on academic matters”.

        Source

        Erickson’s previous position was that of “executive vice president and provost”. I find it difficult to believe that in such a position he would have been kept out of the loop during the course of the Mann whitewashes enquiries.

        Source

    • Posted Nov 17, 2011 at 11:50 PM | Permalink | Reply

      Another potential Penn State person: Karl Goeke, referenced with Neisworth here: http://www.therepublic.com/view/story/FBC-PENNSTATE-PROFESSOR_6604813/FBC-PENNSTATE-PROFESSOR_6604813/ was publishing with Neisworth with a Penn State address here: http://www.sciencedirect.com/science/article/pii/0005791675900191

  6. thisisnotgoodtogo
    Posted Nov 17, 2011 at 6:09 PM | Permalink | Reply

    http://ac360.blogs.cnn.com/

    Anderson Cooper continues his investigative reporting on Penn State.

  7. Speed
    Posted Nov 18, 2011 at 8:01 PM | Permalink | Reply

    Penn St. faculty wants independent investigation

    Faculty members at Penn State on Friday called for an independent investigation of how the university handled allegations of child sexual abuse, and the school indicated that may be forthcoming.

    The faculty Senate endorsed a resolution asking for an investigation to be led by a committee whose chair has no links to Penn State. The resolution also called for a majority of the group’s members to have never been affiliated with the university.

    Penn State has faced criticism since announcing last week that an internal investigation would be led by two university trustees …

    Fresh eyes: The Penn State inquiry needs outside perspective

    The special Penn State committee being formed to investigate circumstances that led to one of the worst scandals to hit a university must be above reproach.

    So far, though, Penn State officials have refused to commit to appointing members who are completely independent of the university. It is essential that they make individuals who have no ties to Happy Valley a part of this process.

  8. DHB
    Posted Nov 19, 2011 at 3:30 AM | Permalink | Reply

    Even if Penn State was not exempt from the law, it still could not release employee information, which would include emails and other conversations. This is true for any government employee even if the state has the open records laws. Idaho is open records but they had to go to court to be able to release information (records, evaluations, emails, face-to-face meeting information,etc.) on a former employee that shot and killed a student. So, even if Penn State was not exempt, they would not have gotten any information – only that the person had worked there, what their position was, their salary, and other basic information that is released yearly anyway.

    • P. Solar
      Posted Nov 19, 2011 at 8:24 AM | Permalink | Reply

      >>
      (records, evaluations, emails, face-to-face meeting information,etc.)
      >>

      I think you are confounding very different things here. Employer records, evaluations etc are rightly protected information.

      Emails sent from govt research accounts (as opposed to a persion’s private account) are FIOA-able, they are not “private” however much some people wish they were.

      • DHB
        Posted Nov 19, 2011 at 1:58 PM | Permalink | Reply

        I am not confounding different things. What the media wants is emails, conversations between employees which IS protected. Here is, in fact, the exemption for ALL agencies under the PaFOIC – “(16) A record of an agency relating to or resulting in a criminal investigation, including:” Here is the website if you would like to see it yourself (http://www.openrecordspa.org/rtk.html#708). As I stated, I also know this because NONE of the emails, conversations, etc. were allowed to be released from the U of Idaho PROFESSOR until a court order said they could and all those emails were from the school email address. I don’t know what you consider a government research account, but I do not believe university emails are considered this type as many people at universities do not participate in research (including the athletic department – not sure when any of them ever did any government research!). These are considered personal as students and professors exchange information about classes, grades, problems, etc. in emails as well as faculty exchange information about departmental meetings, faculty searches, etc. that are all confidential and not related to research whatsoever.

        • Posted Nov 19, 2011 at 3:52 PM | Permalink

          This is probably relevant under Exceptions as well:

          (14) Unpublished lecture notes, unpublished manuscripts, unpublished articles, creative works in progress, research- related material and scholarly correspondence of a community college or an institution of the State System of Higher Education or a faculty member, staff employee, guest speaker or student thereof.

  9. John M
    Posted Nov 19, 2011 at 10:13 AM | Permalink | Reply

    This week’s Sports Illustrated has a series of articles. Those familiar with SI understand that, while focused on sports, there is often broader meaning in the stories, especially the weekly features. These articles refer multple times to the Penn State culture as being “insular” and the University being “isolated”.

    Examples:

    For a prominent university, Penn State is remakably isolated, nestled in the hinterlands of Pennsylvania, six hours from the nearest conference rival and three hours from a major city. (As many learned last week, the impenetrability is heightened by a status that exempts PSU from meanifngful state open-records laws. Many documents related to the Sandusky case, such as e-mails between university officials, are not subject to public disclosure.)

    The idyllic physical setting and the familial spirit of Penn State cut another way: It is a deeply insular place with concentrated power.

    In sorting through the debris, I noticed that [interim football coach] Bradley and new president Rodney Erickson used the same word: transparency, as in an e-mail Erickson sent to the Penn State community last Friday: “I am outlining my promise to the Penn State community, which includes the naming of an ethics officer and a commitment to transparency as the Unversity moves forward.”

    Lots of luck to academics trying to argue that Universities are somehow “special” and their “important work” needs to be shielded from rules that apply to everyone else.

  10. P. Solar
    Posted Nov 19, 2011 at 11:15 AM | Permalink | Reply

    “transparency as the Unversity moves forward.”

    Oh , I like PR. It’s behind us, let’s move forward.

    I think they may find a little “transparency” may be required of them with regards to the past as well.

    “naming of an ethics officer” . Who will dare rape babies in the future if they know there is a “named” ethics officer on campus.

    Let ‘em have it Leroy!

  11. thisisnotgoodtogo
    Posted Nov 20, 2011 at 2:04 AM | Permalink | Reply

    http://ac360.blogs.cnn.com/2011/11/19/video-keeping-them-honest-charitys-files-missing-officials-say/?hpt=ac_mid

    “Second Mile” records missing…

  12. David Smith
    Posted Nov 20, 2011 at 5:15 PM | Permalink | Reply

    More on Penn State’s culture of secrecy –

    http://www.centredaily.com/2011/11/20/2993037/spotlight-shines-on-penn-state.html

    • Posted Nov 20, 2011 at 6:20 PM | Permalink | Reply

      To quote

      From Penn State’s athletic department to the halls of Old Main, the university has long sought to control the public’s access to information about its inner workings…

      This control of information is abetted by a certain insularity, reflected in the unusually long tenures of the leaders at the center of the Jerry Sandusky sex abuse scandal — Spanier and Paterno, Athletic Director Tim Curley and Senior Vice President for Finance Gary Schultz. Curley and Schultz — who both face criminal charges and have stepped down from their positions — are also Penn State graduates…

      This culture may not have caused the sex abuse scandal. But some have suggested that it may have played a role in preventing suspicions about Sandusky from coming to light.

      An analysis by the Chronicle of Higher Education found that insularity extends to the school’s administrative offices. Ten of the 18 members of Penn State’s President’s Council, which comprises the central administration, have been associated with the university for more than 15 years. It found that this insularity is further reflected in the education of the administrators, at least seven of whom have degrees from Penn State or have completed course work there.

      The trustees appointed longtime Penn State insiders to succeed Spanier and Paterno. The new president, Rod Erickson, joined Penn State’s faculty in 1977 and has been working out of Old Main since 1995…

      Terry Mutchler, executive director of the Pennsylvania Office of Open Records, said she always found Spanier’s vigorous fight to keep the records private odd. Now… she said she can’t help wondering… “was it driven by the explosive investigation that’s been lurking right behind the scenes here?” …

      Mutchler acknowledges that most every organization wants to protect its brand. But she said history has shown Penn State goes to great lengths to protect its image. Even 20 years ago when she was a reporter for the student newspaper, The Daily Collegian, she said, “We were fighting tooth and nail” to get basic information. “There is a very real tight grip. I don’t think there is a culture of openness. I think it’s 180 degrees the other way.”

      When one then adds the stories (noted above) of other former Penn State professors Kyle and Neisworth to those of Lasaga and Sandusky, there really is a huge case for concern about the accountability of this university.

    • Posted Nov 21, 2011 at 3:13 AM | Permalink | Reply

      Some very telling quotes in this article, IMHO, particularly (my bold added):

      “It’s like having a family, and you get protective of that,” [Todd Turner, former athletic director at the University of Connecticut, North Carolina State University, Vanderbilt University and the University of Washington] said. “You cherish that. With that you become insulated from what’s going on around you in the world. There is no reason to question anything because it’s always worked.

      and:

      Terry Mutchler, executive director of the Pennsylvania Office of Open Records, said she always found Spanier’s vigorous fight to keep the records private odd. Now with the growing scandal engulfing her alma mater, she said she can’t help wondering whether Spanier’s testimony was solely based on a desire to protect the integrity of the school and its donors.

      [...]

      Mutchler {…] acknowledges that most every organization wants to protect its brand. But she said history has shown Penn State goes to great lengths to protect its image.

      Even 20 years ago when she was a reporter for the student newspaper, The Daily Collegian, she said, “We were fighting tooth and nail” to get basic information.

      “There is a very real tight grip. I don’t think there is a culture of openness. I think it’s 180 degrees the other way.”

      • Speed
        Posted Nov 21, 2011 at 6:17 AM | Permalink | Reply

        Obstruction of justice

        The crime of obstruction of justice, in United States jurisdictions, refers to the crime of interfering with the work of police, investigators, regulatory agencies, prosecutors, or other (usually government) officials.

        Obstruction charges can also be laid if a person alters or destroys physical evidence, even if he was under no compulsion at any time to produce such evidence. Often, no actual investigation or substantiated suspicion of a specific incident need exist to support a charge of obstruction of justice.

        Conspiracy

        In the criminal law, a conspiracy is an agreement between two or more persons to break the law at some time in the future, and, in some cases, with at least one overt act in furtherance of that agreement.

  13. Salamano
    Posted Nov 21, 2011 at 11:22 AM | Permalink | Reply

    It looks like Louis Freeh (former FBI director) is being tapped to head the PSU inquiry into the allegations, and extending it back to 1975.

    I guess it’s a good thing that this scandal did NOT involve academic misconduct, because then it would have been shoved under the rug internally for sure :(

  14. foia
    Posted Nov 22, 2011 at 4:09 AM | Permalink | Reply

    http://files.sinwt.ru/download.php?file=25FOIA2011.zip

    • Posted Nov 22, 2011 at 5:11 AM | Permalink | Reply

      Err wow!

      Thank you !!

      @mosh, where’s your ‘all your base’ youtube link!!

    • Posted Nov 22, 2011 at 5:55 AM | Permalink | Reply

      3334 is interesting for Steve I would have thought:

      date: Tue, 21 Jul 2009 13:22:15 +0100
      from: “Palmer Dave Mr
      subject: McIntyre EIR request (FOI_09-44; EIR_09-03) – Draft response
      to: “Jones Philip Prof , “Mcgarvie Michael Mr

      Phil/Michael,
      A draft response along the lines discussed yesterday. I would expect an almost immediate
      appeal of this decision by Mr. McIntyre.

      Phil, as your concern is the publication of the requested information, I wonder if a
      possible alternative is to release it but place conditions on it’s use. This will ONLY
      work if UEA has some rights in the data itself or in the database. ‘Copyright’ in the
      contents of a database would require some personal creative input by ourselves to the data
      or database that would render it different from preceding external versions and
      ‘original’.

      However, even if the contents aren’t ‘original’, there is a ‘database right’ where the
      contents of the database are assembled as the result of substantial investment in
      obtaining, verifying, or presenting it’s contents. It is the framework, not the contents,
      that attracts the rights. These rights exist for 15 years from the completion of the
      database BUT any substantial change to contents will ‘renew’ the database rights for
      another 15 years. The owner of database rights has the right to prevent the extraction or
      reuse of all or a substantial portion of the database.

      There is ‘fair dealing’ in database rights to the extent that anyone has a right to extract
      & reuse an insubstantial portion of the database (not really defined in law but it’s very
      small) for any purpose, or where the portion is substantial, extract and use data for
      non-commercial research or private study. What can’t be done is re-issuing this information
      to the public under a different guise.

      The upshot of all of this is that, if we have a ‘database right’ in this information, then
      we can release it BUT insist on our exclusive right to re-use the information – BUT the
      issue is actually ‘enforcing’ those rights…… more difficult in practice than in law or
      theory….

      Just thought I would proffer this as an option in place of the refusal and the inevitable
      appeal.

      Cheers, Dave

      <>

      ____________________________
      David Palmer
      Information Policy & Compliance Manager
      University of East Anglia

      Attachment Converted: “c:\eudora\attach\Response_letter_DRAFT2.doc”

    • Posted Nov 22, 2011 at 6:01 AM | Permalink | Reply

      Checked for viruses, malware, comes out clean.

      It looks as if another miracle has happened

      • Posted Nov 22, 2011 at 6:03 AM | Permalink | Reply

        Yep, I scanned it too. Clean as far as I can see. It’s the real deal! :)

    • Posted Nov 22, 2011 at 6:16 AM | Permalink | Reply

      Re: foia (Nov 22 04:09) (that’s nice to include!)

      The last date I can find for an email (using regex search) is 13 Nov 09, on which there are two:

      1046.txt
      date: Fri Nov 13 08:32:28 2009
      from: Phil Jones
      subject: Re: Land/Ocean
      to: Tom Wigley

      4027.txt
      date: Fri Nov 13 08:22:13 2009
      from: Phil Jones
      subject: Re: [Fwd: Your Submission]
      to: Tom Wigley

      Initial hand search, so prone to error. But it looks from this to be a further selection from the same archive taken from CRU, the first selection being released on another .ru server on 17 Nov 09.

      • Posted Nov 22, 2011 at 6:19 AM | Permalink | Reply

        Richard: One of the items in the docs folder is dated today!

        ‘statement.doc’

        • Posted Nov 22, 2011 at 6:23 AM | Permalink

          But all the ‘last modified’ dates have been set to 01-01-2011

        • Posted Nov 22, 2011 at 6:28 AM | Permalink

          Yep, I’ve been searching inside the emails, using TextMate’s regex search so far.

        • Posted Nov 22, 2011 at 6:26 AM | Permalink

          And yet, reading on:

          Attached is a draft Statement that has been informally drawn up by Joe Alcamo, Rob Swart and Mike Hulme working in Europe on climate issues. Its main purpose is to bolster or increase support for controls of emissions of greenhouse gases in European countries in the period leading up to Kyoto.

          Doesn’t sound like today’s draft in Word somehow!

        • Posted Nov 22, 2011 at 6:43 AM | Permalink

          Good point. Strange that today’s date appears on the text of the document itself.

        • Posted Nov 22, 2011 at 6:48 AM | Permalink

          Attempts to ‘clean’ Word to cover one’s traces can have unintended side-effects.

        • Hyperthermania
          Posted Nov 22, 2011 at 6:51 AM | Permalink

          The date at the top of statement.doc is a script, it will just insert your current system date when you open the file.

      • Posted Nov 22, 2011 at 6:22 AM | Permalink | Reply

        I have to say that the start of the README.txt:

        /// FOIA 2011 — Background and Context ///

        “Over 2.5 billion people live on less than $2 a day.”

        “Every day nearly 16.000 children die from hunger and related causes.”

        “One dollar can save a life” — the opposite must also be true.

        “Poverty is a death sentence.”

        “Nations must invest $37 trillion in energy technologies by 2030 to stabilize
        greenhouse gas emissions at sustainable levels.”

        Today’s decisions should be based on all the information we can get, not on
        hiding the decline.

        This archive contains some 5.000 emails picked from keyword searches. A few
        remarks and redactions are marked with triple brackets.

        The rest, some 220.000, are encrypted for various reasons. We are not planning
        to publicly release the passphrase.

        We could not read every one, but tried to cover the most relevant topics such
        as…

        and the timing in 2011 suggest strongly to me a motive to ‘derail Durban’, for the very best of reasons I might add, just as FOIA in 2009 is now best seen as a opportunistic act to take the wind out of Copenhagen.

    • Salamano
      Posted Nov 22, 2011 at 6:27 AM | Permalink | Reply

      I wonder if this is from Anonymous? They often include social justice causes as part of their hacking justifications.

      The readme.txt implies causes larger than just a disgruntled or sympathetic worker on the inside.

  15. kim
    Posted Nov 22, 2011 at 8:35 AM | Permalink | Reply

    The Furies aroused
    By the usual suspect.
    It’s for the children.
    ==========

  16. Posted Nov 22, 2011 at 9:49 AM | Permalink | Reply

    I’m trying to download the files, but I’m running into a security code that needes input ….. Can anyone help in this regard?

    [RomanM: The "security code" should be just to the left of the box in which you are supposed to enter it. It is apparently there to prevent bots from downloading files.]

    • Posted Nov 22, 2011 at 10:06 AM | Permalink | Reply

      Thanks…. doesn’t appear to be working for me…. :-(

    • Posted Nov 22, 2011 at 10:09 AM | Permalink | Reply

      Well, its working now….. probably just too many downloading at once….. thanks again! :-)

  17. Mike
    Posted Nov 22, 2011 at 11:03 AM | Permalink | Reply
  18. Dan from Ohio
    Posted Nov 23, 2011 at 11:53 AM | Permalink | Reply

    Michael Mann does not belong at Penn State, he belongs in the State Penn

    • Mark F
      Posted Nov 23, 2011 at 12:08 PM | Permalink | Reply

      Mann has sued people for saying that – be very careful.

  19. johnb
    Posted Dec 1, 2011 at 10:40 PM | Permalink | Reply

    Interesting to see that Penn State has been purchasing several .xxx domain names. You can’t make that stuff up.

    http://www.foxnews.com/us/2011/11/30/penn-state-buys-adult-domain-names-to-block-usage/

5 Trackbacks

  1. By New Climategate Emails « Climate Audit on Nov 22, 2011 at 8:00 AM

    [...] climategate dossier here. See notice from “FOIA” here and subsequent comments as well as discussion at Jeff Id [...]

  2. [...] Audit Nov 22, 2011 at 4:09 AM | Permalink (Canada, Eastern = 09:09 [...]

  3. [...] http://climateaudit.org/2011/11/16/anderson-cooper-on-penn-state-secrecy/ [...]

  4. [...] http://climateaudit.org/2011/11/16/anderson-cooper-on-penn-state-secrecy/ [...]

  5. [...] November 21st to 23rd inclusive. At 4:09 am on November 22, someone calling themselves FOIA made a comment on McIntyre’s blog. It consisted solely of a link to a zip file posted online at a Russian web [...]

Post a Comment

Required fields are marked *

*
*

Follow

Get every new post delivered to your Inbox.

Join 3,129 other followers

%d bloggers like this: