Exponent’s Transients: Bodge or Botch?

In my first writeup, I observed that Exponent’s Logo transients appeared to be bodged too high, even with their unwarranted and adverse use of 67 deg F initialization (Exponent’s “temperature trick”). In today’s post, I’ve taken a closer look at the seemingly questionable calculation of the transients at 67 deg F, showing that the Patriot transients make sense only if initialization for the transients purporting to show Logo Gauge initialization were not actually initialized at 12.5 psi using the Logo Gauge (as stated and as is the purpose of the diagram).  My reverse engineering shows that the Patriot dry transient in Figure 27 only makes sense if the Logo Gauge read 12.81 psi at initialization or if the Master Gauge (not the stated Logo Gauge) was erroneously used for initialization.  If I’m correct, this is a very significant error –  a botch, rather than a bodge – for which one would expect a prompt corrigendum, if not retraction, of the corresponding calculations.  In a postscript to today’s post, I’ve attached a note on conversion from Logo and Non-Logo Gauge scale to correctly calibrated Master Gauge scale.

Figure 27 – Simulations with Logo Initialization
The critical simulations pertain to initialization with the Logo gauge. These simulations yield the transients in Figure 26, 27 and 30, that are used for comparison of “models” and observations. As previously noted, Exponent carried out these simulations at 67 deg F – the temperature most disadvantageous to Patriots. Their reasons for this adverse assumption were poorly supported and, in my opinion, are completely invalid.  I’ve also observed a puzzling discrepancy between the transients shown in these figures and calculations using the Ideal Gas Law.

First, here is an annotated version of Wells Report Figure 27, showing dry (solid) and wet (dashed) transients for Colt (blue) and Patriot (red) footballs, each supposedly initialized at 13.0 and 12.5 psig at 67 deg F using the Logo gauge. In each case, I’ve shown my reverse-engineered estimates for the actual pressure reading (Logo Gauge) at initialization in order to yield the reported dry transients: 12.81 psig (Patriot) and 13.26 psig (Colt) – 0.31 and 0.26 psi respectively above the Logo readings said to have been used.  In each case, I’ve shown the location of transients displaced downwards by these amounts.  I’ve also shown (solid dots) the Logo average half-time (converted to Master scale using Exponent’s formulas), each coinciding with the horizontal lines in the diagram.  Below the diagram, I’ll explain these calculations and speculate on how the discrepancy may have occurred.

figure27_annotated

Figure 1. Annotation of Wells Report Figure 27 as explained in text.

The solid dots showing the average Logo half-time measurements converted to Master scale (both shown at plausible times( confirm that the conversion formulas have been accurately implemented.

I digitized the Colt dry transient (solid steelblue) and fit it with a negative exponential to an asymptote. The corresponding y-intercept is 11.90 psig (Master).  The pre-game pressure (Master) at 67 deg F initialization required to yield this pressure at 48 deg F ambient is 12.90 psig (Master), which, in turn, is equivalent to 13.26 psig (Logo) using Exponent’s conversion formula.   At  the stated initialization pressure of 13.0 psig (Logo) at 67 deg F, the corresponding y-intercept is 11.67 psig, about 0.23 psi lower than the y-intercept in Figure 27.  A downward translation of the dry transient by this difference of 0.23 psi yields the 13.0 psig transient shown in thicker blue (together with wet transient a further 0.45 psi lower in dashed blue overlapping the Patriot dry transient).

Similarly, the y-intercept of the digitized Patriot dry transient in Figure 27 is ~11.43 psig. The pre-game pressure (Master) at 67 deg F required to yield this pressure at 48 deg F is 12.47 psig (Master), equivalent to 12.81 psig in Logo scale according to Exponent’s conversion formula, ~0.31 psi greater than the 12.5 psig (Non-Logo) said to have been used.  A downward translation of the Patriot dry transient by this difference of 0.31 psi yields the 12.5 psig transient shown in thicker red (together with wet transient a further 0.45 psi lower in dashed red.)   Note that the Figure 27 Patriot transient is consistent with it being calculated at 12.5 psig initialization using the Master Gauge but not with 12.5 psig initialization using the stated Logo Gauge.  Similarly, the Figure 27 Colt transient is consistent with 12.9 psig initialization using the Master Gauge, but not with 13.0 initialization using the Logo Gauge.  This seems like a gross error on Exponent’s part. I’ve attached a detailed script documenting my calculation.  Using 71 deg F initialization, all transients move lower still.

Figure 25 – Simulations with Non-Logo Initialization

For comparison, the corresponding calculations for the Non-Logo simulations (Figure 25) are analysed identically below. In this case, the transients in Figure 25 are consistent with Colt calculations based on 13.1 psig (Non-Logo; 13.05 Master), as opposed to the stated 13.0 psig, and Patriot calculations based on 12.66 psig (Non-Logo; 12.61 Master), as opposed to the stated 12.5 psig, with re-calculate transients translated downwards by a corresponding amount as shown below.

figure25_annotatedFigure 2. Annotation of Wells Report Figure 25 (Non-Logo) as explained in text.

The wet differentials for both Colts and Patriots in this figure (~0.2 psi) are considerably less than the wet differential shown in the Logo diagram (~0.45 psi).  For reference, wet differentials (dashed lines) using this larger differential are also shown in this figure.  As in Figure 1, the average Non-Logo half-time measurements converted to Master scale are shown (solid dots) at plausible times.

Discussion

Exponent clearly stated that its Figure 27 simulations were initialized with the Logo Gauge rather than the Master Gauge:

In recognition of the remaining uncertainty as to which gauge was used to measure the footballs pre-game and in the interest of completeness, similar tests were run using the Logo Gauge. The Logo Gauge was used to set the pressure of two balls to 12.50 psig (representative of the Patriots) and two balls to 13.00 psig (representative of the Colts). From each set (corresponding to each team), one ball remained dry while exposed to the game temperature and the other was wet.

The above Figure 1 shows that this simply isn’t possible. The correctness of this observation can be confirmed by re-calling that initialization at 67 deg F and 12.5 psig yielded half-time pressures at 48 deg F ambient of ~11.5 psig, a value mentioned on several occasions in the Wells Report and observable in Table 10.  The Patriot dry transient in Figure 27 is consistent with initialization using the Master Gauge, but not with initialization using the Logo Gauge, as stated and as supposedly the point of the comparison.

This seems like a botch, rather than a bodge, on Exponent’s part and, if so, ought to require a corrigendum, if not retraction.

These differences of 0.3 psi may not seem like much, but the amount in dispute is only ~0.3 psi they are highly material.  Also note that any lowering of the transients typically increases the time window in which observations are consistent with transients.   Using 71 deg F initialization, all transients move lower still.

Postscript: Conversion to Master Gauge

Before comparing “models” to observations, there are some thorny instrumentation problems that need to be addressed first. Not only was the Logo Gauge biased relative to the Non-Logo Gauge, but it experienced very large drift during Exponent’s studies.  Reading between the lines of the report, it looks like they didn’t think about intra-study drift until after it had occurred, very much complicating and somewhat, in my opinion, compromising their conversion of gauge readings to correct (Master Gauge) pressures.

As a preamble, the gap between Logo and Non-Logo gauge measurements at half-time and post-game averaged ~0.38 psi, with no discernible trend between 10.5 and 13.25 psig – see the black +-signs in the figure at right below (overplotted onto an excerpt from Wells Report Figure 12.)  This ~0.38 psi Logo bias can be seen in Figure 11 of the Wells Report (see excerpt in left panel below) as the bias at 13 psi in the first (Initial – lightblue) tests; the corresponding test for the Non-Logo Gauge showed no bias. The left panel diagram shows that the Logo Gauge bias had increased to ~0.75 psi by the time of the Final test, with the Non-Logo Gauge now biased low by ~0.15 psi.

Figure 12 of the Wells Report (excerpt in right panel) shows two calibrations (V1, V2) over the Logo and Non-Logo gauges over pressure ranges.   Both the V1 and V2 calibrations show considerably larger differences between Logo and Non-Logo readings than actually observed on Game Day, and thus even the V1 tests appear to already experience drift from Game Day readings.

Exponent presented formulas for conversion from Logo and Non-Logo scales to Master scale, with the formulas said to have been calculated from an “early” set of measurements.  The differences from Master scale arising from these formulas is shown as the dotted black (Logo) and dotted green (Non-Logo) lines in the right panel.  The gap between the lines given by the formulas is noticeably less than the consistent ~0.38 psi observed gap on Game Day, strongly suggesting that some drift had already taken place by the time that Exponent calculated its calibration formulas.  The failure of their conversion formulas to preserve this observed difference is a real count against this aspect of their technical work and a frustration to comparative analysis.

figure_11_excerpt

figure_12_annotated

Figure 3. Left – excerpt from Wells Report Figure 11, showing Logo and Non-Logo gauge readings in successive tests: note drift upwards of Logo Gauge. Right – excerpt from Wells Report Figure 12.  See text for further explanation.

Update: August 5, 2015
he error discussed in this post was also discussed by Back Picks here: http://www.backpicks.com/2015/07/25/deflategate-exponents-bias-and-the-master-error/. They illustrated Exponent’s error:

41 Comments

  1. Steve McIntyre
    Posted Jun 29, 2015 at 12:05 PM | Permalink
    #SCRIPT JUNE 29
    #requires R-package png
     # install.packages("png")
    #IDEAL GAS LAW RELATIONS  pV=nRT
    pascal= 6894.76 #pascals per 1 psi
    R= 8.31441 #gas constant J K-1 mol-1 
    Vf= .004237 #m3 volume of football 4237 cm3
    cent =function(x) (x-32)*5/9
    fahr = function(x) 9/5*x +32
    coltp=13.0
    bias=.38 #between Logo and Nonlogo Gauge
    
    Nx=function(targetp,pregame=71) (6894.76*(14.7+targetp))*Vf/(R*(273+cent(pregame)))
     #function to calculate moles from target pressure in psig and pregame temperature in deg 
     Nc= Nx(coltp,pregame=71);Nc # 0.3302896
    
    Pf=  function(tem, n=Nc) (n*R*(273+cent(tem))/Vf)/6894.76 - 14.7   #in psi 
      #function to calculate pressure in psig from temperature in deg F and moles 
      Pf(48,n=Nx(coltp,pregame=71)) #11.798
    
    Tf= function(P, n=Nc) fahr( (Vf* ((P+14.7)*6894.76))/(n*R)  -273 )  #in psi 
      #function to calcuate temperature in deg F from pressure in psig and moles
      Tf(11.798,n=Nc)#47.98409  
     
    #CONVERSION FORMULAS TO MASTER GAUGE
    ###################################
    logof=  function(x)  (x+.2846)/1.05   # Logo to Master: Master = Logo + 0.2836 psi/1.05
      logof(c(11.49,12.74))#11.214 12.404
    nonlogof= function(x) (x+.1444)/1.015 #and Master = Non-Logo + 0.1444 ps /1.015
      nonlogof(c(11.11,12.33)) #11.088 12.290
    logoi=function(x) 1.05*x -.2846 #Logo from Master
    nonlogoi=function(x) 1.015*x -.1444 #Nonlogo from master
    
    ##INPUT FIT TO DIGITIZED TRANSIENTS IN FIGURES 25 (Non-Logo) and 27 (Logo) 
    B=read.csv("http://www.climateaudit.info/data/football/transient_coef.csv")
    B$pregame=rep(c(71,67),each=4)
    B$gauge=factor(B$gauge,levels=c("Logo","Non-Logo","Master"))
    
    setwd("d:/2015/football")
    foot=read.csv("http://www.climateaudit.info/data/football/football.csv")
    sim=read.csv("http://www.climateaudit.info/data/football/sim_table14.csv",nrow=6 )
    
    #FIG27 WITH TRANSIENT CALCULATIONS
    dest="d:/temp/temp.dat"
    o=F #controls output file
    ind=seq(0,14,.2)
    library(png)
    download.file("http://www.climateaudit.info/data/football/football_fig27.PNG",dest,mode="wb")
    img27=readPNG(dest) ##"football_fig27.png"
    
    #SHOW FIG27 ANNOTATED FOR BLOG POST
    if(o) png("figure27_annotated.png",w=640,h=560)
    par(mar=c(0,5,2,5))
    if(o) par(list(font.lab=2,font.axis=2,cex.axis=1.5,cex.lab=1.5))
    plot(0,type="n",xlim=c(-.25,14),xaxs="i",ylim=c(10.25,12.8),axes=F,ylab="")
    rasterImage(img27,-3.22,10.18,17.51,12.92)
    title("Re-stated Figure 27")
    mtext(font=2,line=-.45,"67 Deg F Logo Gauge Initialization",cex=.9)
    axis(side=2,at=seq(10.6,12.8,.2),las=1)
    mtext(side=2, "PSIG (Master Gauge)",line=3.5,font=2)
    axis(side=4,at=logof(seq(11,13,.2)), labels=seq(11,13,.2), las=1)
    mtext(side=4, "PSIG (Conversion to Logo Scale)",line=3.5,font=2)
    
    #Colt
    k=7
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)
    lines(ind,f(ind),col=4,lwd=1)
    f(0) # 11.90
    g=function(x)  (Pf (48,n=Nx(x,pregame=67))- f(0))^2    #46.12
    P0=optimize(g, c( 12,14))$minimum;P0  #12.898
    logoi(P0) # 13.25807
    points(0, Pf (48,n=Nx(logof(13),pregame=67)),pch="+",col=4,cex=2)
    delta=f(0)- Pf (48,n=Nx(logof(13),pregame=67));delta # 0.2369
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)-delta
    lines(ind,f(ind),col=4,lwd=4)
    lines(ind,f(ind)-.45,col=4,lwd=2,lty=2)
    points(8,logof(mean(foot$Logo[12:15])),pch=19,cex=2,col=4)
    text(3.59,11.94,col=4,pos=4,font=2,"13.0 psi")
     text(3.48,12.2,col="steelblue",pos=2,font=2,"13.26 psi")
    
    #Patriot
    k=5
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)
    lines(ind,f(ind),col=2,lwd=1)
    f(0) # 11.49
    g=function(x)  (Pf (48,n=Nx(x,pregame=67))- f(0))^2    #46.12
    P1=optimize(g, c( 12,14))$minimum ;P1 #12.4702
    P1+bias #  12.8502
    logoi(P1) # 12.80911
    Pf (48,n=Nx(logof(12.5),pregame=67)) # 11.20
    delta=f(0)- Pf (48,n=Nx(logof(12.5),pregame=67)) ;delta# 0.2837
    points(0, Pf (48,n=Nx(logof(12.5),pregame=67)),pch="+",col=2,cex=2)
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)-delta
    lines(ind,f(ind),col=2,lwd=4)
    lines(ind,f(ind)-.45,col=2,lwd=2,lty=2)
    points(3.75,logof(mean(foot$Logo[1:11])),pch=19,cex=2,col=2)
    text(3.2,11.49,col=2,pos=4,font=2,"12.5 psi")
    text(2.7,11.73,col="red4",pos=4,font=2,"12.81 psi")
    
    if(o) dev.off()
    
    ##FIGURE 25 ANNOTATION
    download.file("http://www.climateaudit.info/data/football/football_fig25.PNG",dest,mode="wb")
    img25=readPNG(dest) ##"football_fig25.png"
    
    if(o) png("figure25_annotated.png",w=640,h=560)
    par(mar=c(0,5,2,5))
    if(o) par(list(font.axis=2,font.lab=2,cex.axis=1.5,cex.lab=1.5))
    plot(0,type="n",xlim=c(-0.12,14),xaxs="i",ylim=c(10.25,12.9),axes=F,ylab="")
    rasterImage(img25,-3.23,10.225,17.61,12.92)
    title("Figure 25 Annotated")
    mtext(side=3,font=2,"71 deg F Non-Logo Initialization",line=-1)
    axis(side=2,at=seq(10.6,12.8,.2),las=1)
    mtext(side=2, "PSIG (Master Gauge)",line=3.5,font=2)
    #axis(side=2,at=seq(10.6,12.8,.2)-del25,labels= seq(10.6,12.8,.2),las=1)
    axis(side=4,at=nonlogof(seq(10.6,12.8,.2)), labels=seq(10.6,12.8,.2), las=1)
    mtext(side=4, "PSIG (Conversion to Non-Logo Scale)",line=3.5,font=2)
    
    #Colt
    k=3
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)
    lines(ind,f(ind),col=4,lwd=1)
    f(0) # 11.85
    g=function(x)  (Pf (48,n=Nx(x,pregame=71))- f(0))^2    #46.12
    P0=optimize(g, c( 12,14))$minimum;P0  #13.05
    nonlogoi(P0) #13.10165
    P1=Pf (48,n=Nx(nonlogof(13),pregame=71)) #11.75114
    delta=f(0)- P1 ;delta#0.09580259
    points(0, Pf (48,n=Nx(nonlogof(13),pregame=71)),pch="+",col=4,cex=2)
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)-delta
    f(0) #11.63
    lines(ind,f(ind),col=4,lwd=4)
    lines(ind,f(ind)-.45,col=4,lwd=2,lty=2)
    points(8,nonlogof(mean(foot$Nonlogo[12:15])),pch=19,cex=2,col=4)
    text(3.09,11.99,col=4,pos=4,font=2,"13.0 psi")
     text(2.98,12.12,col="steelblue",pos=2,font=2,"13.1 psi")
    
    #Patriot
    k=1
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)
    lines(ind,f(ind),col=2,lwd=1)
    f(0) # 11.42815
    g=function(x)  (Pf (48,n=Nx(x,pregame=71))- f(0))^2    #46.12
    P0=optimize(g, c( 12,14))$minimum;P0  #12.61252
    nonlogoi(P0) #12.66
    P1=Pf (48,n=Nx(nonlogof(12.5),pregame=71));P1 #11.27989
    delta=f(0)- P1 ;delta#0.1482604
    points(0, Pf (48,n=Nx(nonlogof(12.5),pregame=71)),pch="+",col=2,cex=2)
    f=function(x) nonlogof(12.5) - (nonlogof(12.5)- Pf (48,n=Nx(nonlogof(12.5),pregame=71)))*exp(-.115*x)
    f(0) #11.63
    f=function(x) B$A[k]+B$B[k]*exp(-B$C[k]*x)-delta
    lines(ind,f(ind),col=2,lwd=4)
    lines(ind,f(ind)-.45,col=2,lwd=2,lty=2)
    points(3.75,nonlogof(mean(foot$Nonlogo[1:11])),pch=19,cex=2,col=2)
    text(3.09,11.53,col=2,pos=4,font=2,"12.5 psi")
     text(3,11.72,col="red4",pos=2,font=2,"12.66 psi")
    
    if(o) dev.off()
    
  2. mpainter
    Posted Jun 29, 2015 at 4:59 PM | Permalink

    This sort or egregious error by Exponent leaves one gaping with astonishment.
    According to Wikipedia, Ted Wells is a partner in Paul, Weiss, Rifkind, Wharton and Garrison, LLP., also known as Paul, Weiss.
    This firm is HQ in NYC and operates eight offices nationally, with ~ 850 attorneys, total. Gross revenue, 2014: $934.5 million.

    They will be in close touch with Exponent concerning these posts by Steve McIntyre, it can be assumed.

    If they do not consult an independent source concerning Steve’s findings here, but rely exclusively on Exponent for determining the questions raised, it will be another marvel in this most interesting affair.

    • MikeN
      Posted Jun 29, 2015 at 5:09 PM | Permalink

      Esponent’s history involves some high profile simulations, including the Challenger explosion.

    • Steve McIntyre
      Posted Jun 29, 2015 at 7:37 PM | Permalink

      They will be in close touch with Exponent concerning these posts by Steve McIntyre, it can be assumed.

      If they do not consult an independent source concerning Steve’s findings here, but rely exclusively on Exponent for determining the questions raised, it will be another marvel in this most interesting affair

      Even if Paul, Weiss were aware of these criticisms – and, even if they are, they could argue that they aren’t- they would presumably argue that they relied on qualified technical consultants and are not obliged to take account of everything on the internet.

      Having said that, it does appear that this particular error is a misrepresentation of the methodology that they used and that they cannot get the result using the stated methodology. Reminds me in a way of Gergis et al.

      I don’t think that you can assume that Paul, Weiss will necessarily do anything.

      • mpainter
        Posted Jun 29, 2015 at 7:56 PM | Permalink

        This cause is extremely “high profile”, in the parlance of the legal profession and Paul,Weiss has something at stake, and perhaps more than its reputation. If they’re not concerned about the questions raised (here and elsewhere) about the reliability of the Exponent study, it would be another marvel, imo.

      • mpainter
        Posted Jul 4, 2015 at 11:46 PM | Permalink

        I have just read that Ted Wells publicly defended (via a conference call)his report and the integrity of his investigation on May 12, I think. One Reisner, another attorney there was quoted on the thoroughness and reliability of the study by Exponent.

        OOPS

        Lots of reports on the web on this.

        • Steve McIntyre
          Posted Jul 5, 2015 at 8:08 AM | Permalink

          I’ve got a question. The form of analysis in this post is leading very quickly to an interesting Bayesian style analysis. To get there, I need some sort of prior – informative or uninformative. I’m wondering what sort of target deflation (distribution) could be presented as an alternative. If Patriots had a deflation plan, would they try to deflate by 2 psi, 1.5 psi, 1 psi, 0.5 psi, 0.1 psi? Has anyone seen any discussion of how much deflation might have been a target on other occasions?

  3. bmcburney
    Posted Jun 29, 2015 at 5:55 PM | Permalink

    Are you also planning to look at the, possibly related, issue of the Patriots’ unmatched success at avoiding fumbles?

    Steve: since this issue wasn’t raised in the Wells Report, no. My interest in the issue is mainly how Exponent and Wells Report handled the matter, rather than investigating all possible theories on the matter – an approach that is comparable to my approach to climate issues.

    If the topic interests you, I noticed a couple of discussions linked below. In the first article http://www.backpicks.com/2015/05/17/fumbling-statistics-and-patriot-trends/, the author observes that during 2007-2014, the Patriots had the least fumbles in the league, with Atlanta second. The difference was 3 fumbles over 7 seasons, which doesn’t seem like a large margin, though you may disagree. In the second article http://www.backpicks.com/2015/05/30/do-the-patriots-fumble-less-at-home/, the author observed that Patriots fumble rate away was almost identical to at home: indeed, the Patriots were the most uniform of all teams. It doesn’t surprise me that the most successful team over the period has the least fumbles, any more than the best tennis players making the fewest errors.

  4. Ripantuck
    Posted Jun 29, 2015 at 7:09 PM | Permalink

    The fact is that the Patriots have been very successful under Belichek It would be most interesting to try to determine exactly what he does differently Instead, it seems that those who were beaten are utterly desperate to come up with excuses. I have never been a Patriot fan, but would love to see the Redskins try to adopt some of Belichek’s unique methods, whatever they may be.

  5. mpainter
    Posted Jun 29, 2015 at 7:10 PM | Permalink

    An article in the Washington Post today by Sally Jenkins http://www.washingtonpost.com/sports/redskins/roger-goodell-has-one-credible-option-pardon-brady-blame-lax-protocols/2015/06/29/b2c12e28-1e76-11e5-bf41-c23f5d3face1_story.html was scathing toward NFL Commissioner Roger Goodell. In effect, it said that Goodell had no option but to undo the mess that he has made.

    No doubt Jenkins has learned of the posts here by Steve.

    The article was preceded by one written a few days ago, likewise critical of Goodell in this Deflategate affair.
    Jenkins claims that Wells (Paul,Weiss), billed $5 million for the worked performed in the 243 page report (legal fees in NYC run upwards of $1,000/hr).
    She gave no source for that very high figure.

    Which brings us to another very interesting question:

    How much money does the NFL owe for an egregiously flawed report? Obviously Goodell meant for the Patriot organization to make a substantial contribution to that fee, via the $ million fine.
    Now it looks as Goodell might have to ask Ted Wells for an “adjustment” on the bill. All so very interesting.

    • Steve McIntyre
      Posted Jun 29, 2015 at 7:52 PM | Permalink

      Unless Wells or Exponent retract some or all of their report, Goodell is in a very difficult situation because any perception that he is going easy on Brady will be viewed with racial overtones – a point emphasized by, for example, Pardon the Interruption’s Mike Wilbon. If Wells and/or Exponent were trying to be objective, then they should issue a corrigendum or retraction of much of their technical work. It wouldn’t be pleasant but, at some point, it is surely their obligation if their report is supposed to be independent, as opposed to a prosecution brief. If it is merely a prosecution brief, then the NFL should not have been making decisions based on it alone in the first place.

      • mpainter
        Posted Jun 29, 2015 at 8:08 PM | Permalink

        Then Goodell is “impaled on the horns of a dilemma”? Poor fellow.

        • Steve McIntyre
          Posted Jun 29, 2015 at 8:29 PM | Permalink

          I sent the following email to Ted Wells:

          While there have been many criticisms of Exponent’s technical analysis, I wish to draw your attention to a material and highly important misrepresentation of methodology in their technical report. The calculation of transients, used in the important analyses of Figures 27 and 30, is described as follows:

          In recognition of the remaining uncertainty as to which gauge was used to measure the footballs pre-game and in the interest of completeness, similar tests were run using the Logo Gauge. The Logo Gauge was used to set the pressure of two balls to 12.50 psig (representative of the Patriots) and two balls to 13.00 psig (representative of the Colts). From each set (corresponding to each team), one ball remained dry while exposed to the game temperature and the other was wet.

          For reasons set out in more detail at Climate Audit (https://climateaudit.org/2015/06/29/exponents-transients-bodge-or-botch/), it is impossible that the bolded statement is a correct description of Exponent’s actual methodology. To arrive at the dry Patriot transient illustrated in Figure 27, the Logo Gauge would have had to have been set to ~12.81 psig. It appears more likely that a different gauge was used to initialize the footballs in the simulations illustrated in these figures. If Exponent had used the stated methodology, they would not arrive at the results illustrated in Figures 27 and 30.

          The misrepresentation is material and warrants an immediate corrigendum, together with retraction of technical analysis depending on this mistake. It is by no means the only statistical error in Exponent’s report, but is somewhat distinguished by it being an actual misrepresentation as opposed to a methodological critique.

          I have considerable experience in statistical analysis. I have made presentations to a panel of the U.S. National Academy of Sciences, a subcommittee of the U.S. House of Representatives and my work has been covered on the front page of the Wall St Journal, by CNN and Fox News and internationally. In 2010, the New Statesman magazine in the U.K. recognized me as one of “50 People Who Mattered” in 2010.
          Regards,
          Stephen McIntyre
          Climate Audit
          http://www.climateaudit.org

        • Salamano
          Posted Jun 30, 2015 at 5:40 AM | Permalink

          I wonder if your avoidance of the phrase ‘more likely “than not”‘ at a crucial juncture in your letter was diplomatically intended. It could certainly have been used a time or two…

        • Bill
          Posted Jul 5, 2015 at 10:09 AM | Permalink

          Steve,
          Did you receive any response to your email to Ted Wells? If so, are you able to share the response and any follow-up communications?

          Steve: no acknowledgement.

      • Salamano
        Posted Jun 30, 2015 at 5:54 AM | Permalink

        Oh, and check out this little nugget, a quote from a judge’s ruling against the Exponent regarding the disclosure of materials pursuant to their scientific methodology in another case:

        http://thornography.weei.com/sports/boston/2015/06/29/wells-report-science-firm-exponent-gets-whacked-by-court-order/

        “[T]he Court cannot allow Exponent to stand in violation of a valid Court order compelling the production of documents which were demanded pursuant to a lawful subpoena and found relevant by the Court,” Judge Stephen A. Stobbs wrote in the June 2, 2015 order. “Methodologically sound science has nothing to fear from full and open disclosure.”

        Hmm… now what other field of science have I heard this kind of back and forth before…

      • Matt Skaggs
        Posted Jun 30, 2015 at 9:41 AM | Permalink

        “If it is merely a prosecution brief, then the NFL should not have been making decisions based on it alone in the first place.”

        If you were presenting to a jury, that statement would go at the end of the summation, amirite? You need a bit of poetry here, like “if the transient doesn’t fit, you must acquit.”*

        *for the British readers and those below the age of 40, see O.J. Simpson murder trial.

  6. MikeN
    Posted Jun 29, 2015 at 9:43 PM | Permalink

    Isn’t the transient something they measured in simulations? How can they calculate a transient?

    Steve: it was measured in simulations. But the simulations are subject to the Ideal Gas Law and ought to be accurately estimated for dry footballs. Their transient is inconsistent at the stated initialization.

  7. editstet
    Posted Jun 29, 2015 at 9:56 PM | Permalink

    A widening gap in readings between the two gauges can be seen in the record for the Patriots balls as they were measured at half-time as well.
    Regarding the fumbles, well, Deadspin did something pointing out problems with the initial analysis. Brady ranks among the fastest quarterbacks in getting the ball out and on punt returns and kickoffs each team uses the same ball. It’s the little things that screw up such a great story:
    http://regressing.deadspin.com/why-those-statistics-about-the-patriots-fumbles-are-mos-1681805710

  8. mpainter
    Posted Jun 29, 2015 at 10:38 PM | Permalink

    Steve,
    John C Hearn
    Chief Professional Responsibility Officer, Paul, Weiss

    Note his office. Perhaps a copy to him would be appropriate:

    jhearn@paulweiss.com

    Steve: good idea. Will do tomorrow.

  9. mpainter
    Posted Jun 30, 2015 at 1:30 PM | Permalink

    Salamano:
    Very interesting link. The judge lowered the boom on Exponent. Fancy that they imagined that they could defy a court order!

    • Steve McIntyre
      Posted Jun 30, 2015 at 2:57 PM | Permalink

      The court case in question appears to be an asbestos liability case. The lawyers are asbestos plaintiff litigators and Exponent has acted for asbestos defendants. “JM Company” is presumably a descendant of Johns Manville. Googling Exponent and asbestos litigation, turns up many interesting items, including schemes to keep scientific data confidential under attorney-client privilege while at the same time publishing the results in academic journals to provide citeability.

      See http://www.publicintegrity.org/2013/10/21/13559/facing-lawsuits-over-deadly-asbestos-paper-giant-launched-secretive-research

      One judge found:
      There’s something extremely smelly about claiming attorney-client privilege for something that is being claimed at the same time as good science,” said Sheila Jasanoff, a professor at Harvard University’s John F. Kennedy School of Government who has written extensively about litigation-driven research. “Legal confidentiality protections should not be placed around good science.”

      The decision prompted an editorial this month in the Annals of Occupational Hygiene, which published two of the Exponent papers funded by Georgia-Pacific. “While these revelations do not in any way prove that the data used in the two Annals papers were fraudulent or that the authors’ conclusions were not legitimately based on the data, they do challenge the principles of free and open scientific inquiry,” chief editor Noah Seixas wrote, noting that the journal was reviewing its conflict-of-interest policies for authors.

      http://annhyg.oxfordjournals.org/content/57/8/963.full.pdf+html

      The court suggests that the research was sponsored with a particular outcome in mind and that the sponsor of the research, without attribution, had significant input to the conduct and reporting of the results.

      I wonder if Exponent and Wells have been involved in the NFL concussion case.

      • mpainter
        Posted Jun 30, 2015 at 4:09 PM | Permalink

        Brad Karl is given as the Paul,Weiss partner who represented the NFL in the concussion case. I could find no references to Exponent in that case. Wikipedia gives some interesting information on Exponent.

        • Posted Jun 30, 2015 at 4:53 PM | Permalink

          From Wikipedia, “Until 1998, it was known as Failure Analysis Associates and its holding company, The Failure Group, Inc., was traded as the symbol FAIL.” Seems appropriate. 😉

        • Steve McIntyre
          Posted Jun 30, 2015 at 6:13 PM | Permalink

          The NFL is represented by Brad S. Karp, Theodore V. Wells Jr., Beth A. Wilkinson and Lynn B. Bayard of Paul, Weiss, Rifkind, Wharton & Garrison in New York, Judy L. Leone and Robert C. Heim of Dechert in Philadelphia and John J. Soroko and Dana B. Klinges of Duane Morris in Philadelphia.

          In the asbestos cases, Exponent appears to have been involved in publishing scientific studies that supported the industry’s cases. Exponent also has a scientist specializing in concussions http://www.exponent.com/william_bussone/

          His work includes analysis of traumatic injuries associated with transportation, industrial, sports, and amusement activities. Mr. Bussone has published research in brain injury biomechanics and amusement ride biomechanics. In his capacity as a Research Associate at the Virginia Tech – Wake Forest Center for Injury Biomechanics at Virginia Tech, performed in vivo analysis of concussions and brain injuries in collegiate football and head accelerations in everyday events

          I haven’t looked at his work, but I wouldn’t be surprised if the conclusions were favorable to the NFL, if one were to extrapolate from Exponent’s asbestos techniques.

          BTW I just watched a local sports talk show in which Luke Willson, tight end for Seattle, was interviewed at length. Willson is originally from southern Ontario. Willson was an extremely engaging young man. They asked him about concussions. He said that Seattle taught styles of tackling without using the head and he thought that the dangers had already been much reduced. Maybe just the optimism of a young guy. He said he was a big believer in playing multiple sports and he was playing squash in the offseason. I immediately paid more attention to him.

      • Follow the Money
        Posted Jun 30, 2015 at 9:43 PM | Permalink

        Oh my…just a cursory look on the web shows all sorts of cases this firm has done with the NFL. Also some NHL too. The refs I believe are direct employees of the NFL—and are chosen by the NFL, not teams or individual owners. There is an utter conflict here in favor of the refs, or in finding their activities not inadequate or negligent, as extensions of the NFL itself.

        Here is info from Attorney Reisner’s bio:

        Successfully represented the National Hockey League in various litigation matters, including defending use of the “Minnesota Wild” trademark by the NHL’s Minnesota franchise … Successfully represented the National Football League in various litigation matters, including winning an injunction against an ambush marketing campaign that infringed NFL trademarks.

    • mpainter
      Posted Jul 5, 2015 at 4:20 PM | Permalink

      This defiance of a court order intrigues me. It is impossible to believe that they have nothing to hide.
      #####

      Speedy et al vs
      3M Corp. et al

      in the Third Judicial District of Illinois

      Plaintiff, age 25, diagnosed with mesothelioma; claims exposure to asbestos as a child.
      Co-defendents include Boeing, United Technologies,Georgia Pacific, Honeywell International, Goodyear Tire and Rubber, others.

      Exponent claimed privilege for certain documents not so classified under Illinois law. Seems to bear on Exponents methodologies in asbestos related studies. The court on May 11 ordered Exponent to deliver the documents to the Plaintiffs. Exponent has defied this order.
      From the ruling of June 2, 2015:

      “Exponent has failed to comply with this court’s order and has indicted that it will continue to disregard this court’s May 11 order”

      So Exponent thinks to appeal. They must have sound reasons for wanting to hide their methodologies. I am reminded of Steve’s FOI fight against Keith Briffa and the CRU.

  10. mpainter
    Posted Jun 30, 2015 at 6:56 PM | Permalink

    Rethinking this affair, it seems likely that Wells would amend the report to read, from “more probable than not” to ” uncertain”, if he got the nod from Goodell, the client. The “prosecution” would simply be dropped.
    There is no question that Goodell needs to re-evaluate his position;he is looking at a lawsuit from Brady supported by the Players Association, and all hinges on dubious science by dubious Exponent.
    I would guess that Goodell will find away to get clear of this mess.

    • mpainter
      Posted Jun 30, 2015 at 7:19 PM | Permalink

      Except for the fact that he p—-d away a reported $5 million of the league’s money.

  11. Grant
    Posted Jul 1, 2015 at 11:42 PM | Permalink

    Have you considered sending this to Daniel R. Marlow – Professor of Physics
    Princeton University? He supposedly advised Wells on the science and seems respected by his peers. He may be someone who is concerned about his credibility – cause I don’t think either Exponent nor Wells do. marlow@princeton.edu

    • mpainter
      Posted Jul 2, 2015 at 1:47 PM | Permalink

      If he advised Wells, he did so for a fee and he thus has a particular point of view in this matter, perhaps.

  12. Posted Jul 2, 2015 at 8:25 AM | Permalink

    Great work steve. Another site has analyzed fumble rates, brady’s performance home vs. road as well as the science and other parts of the wells report etc. here: http://www.backpicks.com/

  13. mpainter
    Posted Jul 6, 2015 at 3:45 PM | Permalink

    Methinks that I perceive another botch in Exponent’s transient curves. I refer to the wet ball transient curves (perhaps this has been raised before). This error can be best seen in the Wells Report fig. 21 (fig 1 in Steve’s analysis).
    Note that the wet ball transient at halftime shows wet ball re-pressurization at the same rate as the dry balls for the first few minutes. This is impossible, if I’m not mistaken. I do not see how wet balls can begin to warm/re-pressurize until they have become dry balls.

    If, for example, it took two minutes for a damp ball to dry, then warming/repressurization should not begin until that point. From that point onward,the transient becomes a dry ball transient. The wet ball transient in Wells fig 21 is incomprehensible when viewed in light of this consideration.

    Shifting the warming/re-pressurization of wet balls by two minutes into the half-time interval will have the effect of shifting measured psi down by 2.5-3 psi, it appears.

    • mpainter'm
      Posted Jul 7, 2015 at 1:40 PM | Permalink

      Correction,
      by 0.25-0.3 psi

  14. MikeN
    Posted Jul 28, 2015 at 10:52 AM | Permalink

    http://www.backpicks.com/2015/07/25/deflategate-exponents-bias-and-the-master-error/

    • mpainter
      Posted Jul 28, 2015 at 2:10 PM | Permalink

      I have just posted a comment at this blog concerning the game day drop in barometric pressure as per Norwood Airport: 7 MB (0.1 psi) from 4 pm to 8 pm, approximately. This should have caused the same in all of the game balls.

  15. John Faulstich
    Posted Aug 5, 2015 at 8:49 PM | Permalink

    If you have not sent to the Patriots, they would certainly have interest. Stacey James, Vice President of Media Relations, staceyj@patriots.com, 508-326-6868 would be a good start

  16. Steve McIntyre
    Posted Aug 6, 2015 at 7:23 AM | Permalink

    The error discussed in this post was also discussed by Back Picks here: http://www.backpicks.com/2015/07/25/deflategate-exponents-bias-and-the-master-error/. They illustrated Exponent’s error:

    • John Faulstich
      Posted Sep 8, 2015 at 8:44 PM | Permalink

      Steve, why don’t you email Mike Florio and talk to him about all of the exponent errors?

  17. John Faulstich
    Posted Sep 2, 2015 at 12:01 PM | Permalink

    Nicely written article http://business.financialpost.com/fp-comment/deflating-deflategate-confusion-about-basic-science-could-have-wrongly-smeared-tom-brady

3 Trackbacks

  1. […] an earlier article,  I pointed out that essential figures in the Exponent report contained (what appeared to be) an […]

  2. […] game temperature and the other was wet. For reasons set out in more detail at Climate Audit (https://climateaudit.org/2015/06/29/exponents-transients-bodge-or-botch/), it is impossible that the bolded statement is a correct description of Exponent’s actual […]

  3. […] contemporary evidence of such a possibility.   A news story shortly after the game (on NFL.com here) reports an interview with D’Qwell Jackson, a Colt cornerback who intercepted a Brady pass in […]

%d bloggers like this: