Exponent’s Trick to Exaggerate the Decline

In an earlier article,  I pointed out that essential figures in the Exponent report contained (what appeared to be) an important misrepresentation: that transients purporting to represent Logo gauge initialization had not really been initialized with the Logo gauge.  The same point was later (and independently) made in a technically oriented sports blog.  Exponent’s misrepresentation (“trick”) exaggerated the “decline” – a word that climate readers will be amused to find in discussion of this topic.  To effectively defend their client,  in my opinion, this was perhaps the most important job for Brady’s technical experts. And, out of all the technical issues, because this issue involved a misrepresentation – be it unintentional or intentional – this was arguably the technical issue with the biggest upside for Brady’s legal team, since misrepresentations have an entirely different legal weight than errors.  In today’s post, I’ll look at the transcript to see how Snyder (the technical expert) and Kessler (the lead lawyer) did on this issue. I’ll also suggest a face-saving solution for Goodell.

Exponent’s Trick

One of the most fundamental technical findings of the Wells Report was the following:

regardless of the assumptions made with respect to the gauges used pre-game and at halftime, the measurements recorded for the Patriots game balls at halftime cannot be entirely explained by the Ideal Gas Law (or variations thereof) when applied to the most likely game conditions and circumstances.

Exponent was aware of the uncertainty over whether pregame measurements of Patriot balls had been made with the Logo or Non-Logo gauge and argued that there was unexplained deflation even with Logo Gauge initialization. The keys to this analysis was their comparison of observations to transients in Figures 27 and 30, both of which, together with Figure 26, were said to “the scenario in which the footballs were set with the Logo Gauge“.

figure 26 caption

figure 27 caption

figure 30 caption

In my post on June 29 entitled Exponent’s Transients: Botch or Bodge, I pointed out that it was simply not possible for Exponent’s transient purporting to show Logo Gauge initialization to have been produced with Logo Gauge initialization and that Exponent had mis-characterized how the figure was produced.  The transients in Figures 26, 27 and 30 were therefore significantly too high relative to Exponent’s description and subsequent use of the figure. I surmised that Exponent had used the Master Gauge for initialization, not the Logo Gauge  to initialize the transients:

The Patriot dry transient in Figure 27 is consistent with initialization using the Master Gauge, but not with initialization using the Logo Gauge, as stated and as supposedly the point of the comparison.  This seems like a botch, rather than a bodge, on Exponent’s part and, if so, ought to require a corrigendum, if not retraction.

About a month later, Back Picks arrived at the same conclusion, stating it as follows:

Only they fail to present an adjusted curve! Figure 26 [SM: which has the same transients as Figure 27] is simply wrong.  The curve shows a dry starting halftime value of over 11.5 PSI for the expected Patriot values. But a Master-adjusted Patriot ball would actually be 12.17 PSI in the pre-game according to Exponent. A dry football is expected to be 11.20 PSI at 48 degrees if it were set at 12.17 PSI in a 67 degree environment in the pre-game, as Exponent is attempting to model. The graph is not master-adjusted, even though Exponent claims it is. It is a clear error and needs to be corrected. [my bold]

Back Picks illustrated the point with the following figure that improved on my rendering. In the corrected figure,  even at the very adverse 67 deg F assumption (the temperature most disadvantageous to Patriots) the average time of Patriot measurement without inconsistency with the transients increased from an upper limit of 2 minutes (as in Exponent Figure 27) to nearly 6 minutes – easily accommodating the observations. With 71 deg F initialization, the time limit increased even further.

Figure 1. Showing the error in Exponent’s Figure 27 (and the same transients are used in Figure 30) from Back Picks.

Once again, this is not an error in a small detail, but an error that goes to the heart of Exponent’s claim that there are issues regardless of which gauge was used for pre-game initialization of Patriot balls.

The Hearing

Brady’s technical expert, Edward Snyder, raised the issue of the Logo Gauge, but inexplicably did not challenge Exponent’s transients in Figures 26, 27 and 30.

In direct, Snyder observed that a pre-game measurement of 12.5 psi with the Logo Gauge equated to a measurement of 12.17 psig using the Master Gauge and, allowing for this inaccuracy, 8 of 11 footballs were in compliance.  (This is an important and essential issue but is not a direct challenge to the Exponent mispresentation of their Figures 26, 27 and 30.)

In cross-examination, Snyder was challenged on Patriot statements that they had tendered balls at 12.5 psi. Snyder (in my opinion, reasonably) pointed out that there was no data on the calibration of the Patriot gauge and there was no basis on which one could say that it might not have had a similar bias to referee Anderson’s Logo gauge.  This is an important issue, but does not touch on Exponent’s misrepresentations of Figures 26, 27 and 30.

Caligiuri

In Kessler’s cross-examination of Caligiuri (Exponent), he failed to confront him with the misrepresentations in Figure 26, 27 and 30.  On occasion, his questions came close to the topic, but he was presumably unaware of the misrepresentations and missed opportunity after opportunity.

On the other hand, Caligiuri was aware of the issue and his answers are instructive. Caligiuri appears to say that they did not do tests with Logo gauge initialization because that was not “consistent with the physical facts”:

Q. Second, do you agree, you saw the criticism number 3 that Dr. Snyder presented, and he indicated that you should have recalibrated the starting pressures through the master gauge because you were comparing those starting pressures to the halftime pressures, which you did to the master gauge. Do you agree that you should have done that?

A. No.

Q. So you think it’s appropriate to take one set of pressures, not do the master gauge and compare it to another set of pressures through the master gauge; that’s your opinion?

A. No. The opinion is, in fact, the 12.5 we used is a master gauge reading. It is —

Q. You didn’t do — for the two balls you tested at halftime, logo or non-logo, you translated to the master gauge, right?

A. Correct.

Q. And for the starting time, you didn’t do any translation to the master gauge, whether it was logo  or non-logo, right? You didn’t make any change?

A. You said “starting time.”

Q. The pre-game measurement, you didn’t do any translation to the master gauge there?

A. The master gauge is 12 and a half percent — 12 and a half psi, and we used 13.0 psi for the Colts.

Q. Did you do any calculation for the pre-game testing to convert the measurements recorded to something in the master gauge?

A. The master gauge conversion, if you convert the 12.5 psi comes from use of the logo gauge pre-game. And that, as Dean Snyder says, is 12.17 percent — 12.17 psi. You can put that in there and you can do Ideal Gas Law calculations, but they are not consistent with the physical facts.

At this point (or earlier), Kessler ought to have confronted Caligiuri with the misrepresentations about Figures 26, 27 and 30, but doesn’t seem to have realized the importance of the issue.  So instead of pointing out and confronting Caligiuri with Exponent’s (false) claims that the transients in Figures 26, 27 and 30 had been “set” with the Logo gauge, Kessler went off into an argument about Exponent’s basis for believing that the Non-Logo gauge had been used – an important topic, but one that could have been pursued after dealing with Exponent’s misrepresentations:

Q. They are not consistent with what physical fact?

A. The fact that if that was happening, then the Patriots gave the referees 12.17 psi balls, below the League minimum.

Q. How do you know that the Colts — and, I’m sorry. How do you know that the Patriots and the referee were both not using something equivalent to the logo gauge?

A. So you’re asking me to assume that all three of these people, the Patriots pre-game, the Colts  pre-game, and Mr. Anderson pre-game all used the  same gauge that were exactly the same amount off?  All the tests —

Q. No.

A. I’m sorry; go ahead.

Q. I am asking the following. You have never seen or tested or looked at the Colts’ gauge or the Patriots’ gauge pre-game, right?

A. That’s correct.

This line of questioning then spiraled off into a discussion of gauges – an important topic, but again without confronting Exponent’s misrepresentation.

While the transcript is not as on point as one would like, it certainly supports the surmise that Exponent calculated the transients in Figures 26, 27 and 30 with pressures set at 12.5 and 13.0 psig using the Master Gauge, not the Logo Gauge (as shown in the figures, which misrepresented the procedure.)

Caligiuri explained why he thought that initialization of transients with the Master Gauge was the “right” procedure, but did not explain why Exponent’s report stated that they had “set” the footballs using the Logo Gauge, when they had actually used the Master Gauge.   Kessler, apparently one of the finest litigation lawyers of his generation, advised by an able and experienced expert, then totally fanned on an essential question much to the detriment of the truth in the matter and to the detriment of his client.  The net result is that Exponent’s misrepresentations were left unchallenged, together with Exponent’s false findings based on these figures.  The incident should be a warning about the vagaries of legal processes even with very competent lawyers.

That Exponent’s misrepresentation lingers on is disquieting.  On the other hand, it’s the sort of thing that could give the parties a basis of face-saving settlement for Goodell at Exponent’s expense.   Suppose that Goodell said something along the following lines:

I have learned that the technical analysis in the Wells Report contained an important and erroneous mis-statement of procedure – in key figures, simulations and transients, said to have been set using the Logo Gauge, were actually set using the Master Gauge. After correcting this error, we can no longer say that there was unexplainable deflation regardless of assumptions on gauges.  While there are disquieting texts, without being sure that there was unexplainable deflation, we can no longer conclude that an offence took place.  We have already announced strict measures to ensure that a similar controversy cannot arise in the future.

Doesn’t that seem like the only sane ending?


14 Comments

  1. Posted Aug 7, 2015 at 11:03 AM | Permalink

    Steve, you literally saved me from writing this post. I found this to be one of the most astonishing exchanges in the appeal, and one of the few times Kessler really whiffed.

  2. MikeN
    Posted Aug 7, 2015 at 2:31 PM | Permalink

    6 minutes average time?
    Exponent calculate 30 seconds to measure each football, which would place the total time for 11 footballs around 6 minutes, allowing for a start time as late as 3 minutes into halftime.
    If they measure in an assembly line fashion with ref 1 measuring ball n+1 while referee 2 is measuring ball n, then total time is just 3 minutes.
    This graph also is in line with the Colts being measured at the end of halftime.

    There is still the issue, which was brought up by one witness, that the Patriots balls are moving in the wrong direction with regards to pressure.

    • Steve McIntyre
      Posted Aug 7, 2015 at 2:55 PM | Permalink

      There is still the issue, which was brought up by one witness, that the Patriots balls are moving in the wrong direction with regards to pressure.

      The “trend” is not significant. Nor, given the noisiness, are the observations necessarily inconsistent with warming.

      • MikeN
        Posted Aug 7, 2015 at 8:27 PM | Permalink

        You can’t say the noisiness means the numbers are consistent with physics, when the noisiness is being used as evidence of tampering.

      • MikeN
        Posted Aug 7, 2015 at 8:30 PM | Permalink

        Would evaporative cooling had been detected during the tests Exponent reported?

    • Steve McIntyre
      Posted Aug 7, 2015 at 2:57 PM | Permalink

      Exponent calculate 30 seconds to measure each football, which would place the total time for 11 footballs around 6 minutes, allowing for a start time as late as 3 minutes into halftime.

      It seems to me that they would have needed more time to reflate and regauge the 11 footballs than to do the original measurements.

      • MikeN
        Posted Aug 7, 2015 at 8:25 PM | Permalink

        They reported deflation could be done in 1 minute 40 seconds, so it’s not difficult.
        I was just pointing out that 3-9 minutes is average of 6.

        • Posted Aug 8, 2015 at 11:04 AM | Permalink

          A couple things:

          1. They tested *blind* deflation. It makes sense that Inflating to a specific level (e.g. ~13 PSI) takes longer because they have to check how much they inflated the ball.

          2. If you follow their estimations (2-4 minutes to set up, 4-5 min to measure the Patriots), the earliest average measurement time would be 4 minutes. The latest would be 6.5 minutes. (Given that those are ballpark estimates, I don’t think it’s impossible to rule out something slightly earlier.)

        • MikeN
          Posted Aug 9, 2015 at 2:36 PM | Permalink

          I wonder who came up with the time estimate. I suspect they did a modified assembly line approach, which would also explain how the measurements got switched at one point, and the total time would be on the low end near 4 minutes.

        • MikeN
          Posted Aug 9, 2015 at 8:43 PM | Permalink

          It was a joke about their analysis of deflation time.
          Plus that was with one person, now they have a group doing the inflation.

  3. Posted Aug 8, 2015 at 2:05 PM | Permalink

    I think that the ‘misrepresentation’ in this case is more likely due to lack of care than intent. I doubt very much that the stats group figured out what was really going on and simply missed it. Of course, court experts find what they are paid to find or you find new experts. Now that it is exposed, and the embarrassment is real, you have uncovered what is likely a much lower grade problem than climategate ‘hide the decline’.

    The title of this article, while fun, minimizes what was done in the climate case where the authors were fully aware of what they were hiding when they did it.

    • Posted Aug 8, 2015 at 2:24 PM | Permalink

      and instead simply missed it

    • Steve McIntyre
      Posted Aug 8, 2015 at 3:23 PM | Permalink

      Yes, the Climategate “hide the decline” was much more cynical. When Goodell used “decline” in his decision, I couldn’t resist the comparison.

      As I understand (and use) the word “misrepresentation”, there can be both “innocent” and “deliberate” misrepresentations. The word itself does not imply intent. In late June, I notified Wells of the error, but they continued to make the misrepresentation in the hearing before Goodell. In the hearing, Exponent took the position that it made no sense to do simulations based on the premise that the Logo gauge was used for pregame initialization, so it’s that much harder to figure out their intent in describing Figure 27 simulations in the way that they did.

      I notified Daniel Marlow today of the error as well. Marlow is apparently a very sincere and thoughtful person. It will be interesting to see whether he’s persuaded by the argument, and, if he isn’t, why he isn’t.

  4. MikeN
    Posted Aug 10, 2015 at 7:04 AM | Permalink

    Ms. Mills does not believe that she has paper copies of potential federal records in her possession. Following our production on August 10, 2015, we have instructed her to delete any and all electronic copies in her possession.

    Given that the documents Ms. Mills produced and will produce to the State Department are the subject of inquiries, Paul, Weiss will maintain an electronic version of the Mills production at our office until we receive additional instructions from you regarding this material.

One Trackback

  1. […] « Exponent’s Trick to Exaggerate the Decline […]

%d bloggers like this: