Letter to Daniel Marlow on Exponent Error

On June 29, I sent a letter to Ted Wells, notifying him of the erroneous description of key figures in the Exponent report, but did not receive any acknowledgement.  In the presumption that Daniel Marlow of Princeton is more likely to be concerned about the erroneous research record (as well as having obligations that the research record be properly presented) I sent him a similar letter today, copying lawyers Daniel Goldberg and Jeffrey Kessler.

Continue reading

Exponent’s Trick to Exaggerate the Decline

In an earlier article,  I pointed out that essential figures in the Exponent report contained (what appeared to be) an important misrepresentation: that transients purporting to represent Logo gauge initialization had not really been initialized with the Logo gauge.  The same point was later (and independently) made in a technically oriented sports blog.  Exponent’s misrepresentation (“trick”) exaggerated the “decline” – a word that climate readers will be amused to find in discussion of this topic.  To effectively defend their client,  in my opinion, this was perhaps the most important job for Brady’s technical experts. And, out of all the technical issues, because this issue involved a misrepresentation – be it unintentional or intentional – this was arguably the technical issue with the biggest upside for Brady’s legal team, since misrepresentations have an entirely different legal weight than errors.  In today’s post, I’ll look at the transcript to see how Snyder (the technical expert) and Kessler (the lead lawyer) did on this issue. I’ll also suggest a face-saving solution for Goodell.
Continue reading

Who “Told” Exponent Not to Consider Switching Scenario?

The transcript of the Brady appeal before Goodell has been released and it’s astonishing to see how the sausage was made.  It raises many issues, one of which I’ll discuss in today’s post. Continue reading

Goodell and Deflategate Science

Yesterday, Roger Goodell released his decision on the Brady appeal.

Most of the early discussion has been about Brady’s destruction of his cell phone. Brady has contested the NFL’s characterization of this incident here (see cover here), saying that he had replaced a broken phone; that they had already told the NFL that Brady was not going to turn over his cell phone and that Brady had no obligation to do so under the labour agreement; that they provided the NFL with records from the carrier of all calls and texts; that he had “never written, texted, emailed to anybody at anytime, anything related to football air pressure before this issue was raised at the AFC Championship game in January”; that Wells already had Jastremski and McNally’s phones (on which there were no communications from Brady until after the AFC Championship game).  More on this below.

My specific interest in the decision was how the scientific issues were dealt with, given that there were serious statistical and scientific defects in the Exponent report.   There isn’t very much in the Goodell decision about the science and statistics. Goodell adopted the Exponent report in total.  It also looks to me like Brady’s side did a totally ineffective job of confronting the Exponent report.

Goodell accepted Exponent’s finding that the full extent of the decline could not “be explained” and that a “substantial part of the decline” was due to tampering. Goodell says that the Brady side submitted “alternative scientific analyses (including the study presented by economists from the American Enterprise Institute)” and, as an expert witness, produced Dean Edward Snyder of the Yale School of Management, described as an “economist who specializes in industrial organization”.    Against them, the “Management Council” produced two Exponent scientists (Caligiuri and Steffey) and the Princeton professor who had originally reviewed the Exponent study.

The salient section is as follows (with two footnotes) :

I find that the full extent of the decline in pressure cannot be explained by environmental, physical or other factors. Instead, at least a substantial part of the decline was the result of tampering….

I took into account Dean Snyder’s opinion that the Exponent analysis had ignored timing… Dr Caligiuri and Dr Steffey both explained how timing was, in fact, taken into account in both their experimental and statistical analysis. They concluded based on physical experiments that timing of the measurements did have an effect on the pressure but that the timing in and of itself could not account for the full extent of the pressure declines hat the Partiot balls experienced.  Dean Snyder, in contrast, performed no independent analysis or experiments, not did he take issue with the results of the Exponent experimental work that incorporated considerations of timing and were addressed in detail in the testimony of Caligiuri and Steffey.

I also considered Dean Snyder’s other two “key findings”, as well as the arguments summarized in the NFLPA’s post-hearing brief, including criticism of the steps taken in the Officials Locker Room at halftime to measure and record the pressure of game balls[1]. I was more persuaded by the testimony of Caligiuri, Steffey and Marlow and the fact that the conclusions of their statistical analysis were confirmed by the simulations and other experiments conducted by Exponent. Those simulations and other experiments were described by Prof Marlow as a “first-class piece of work”.[2]

[1] There was argument at the hearing about which of the two pressure gauges Anderson used to measure the pressure in the game balls prior to the game. The NFLPA and Snyder opined hat Mr Anderson had used the so-called logo gauge.  On this issue, I find unassailable the logic that the Wells Report that the non-logo gauge was used, because otherwise neither the Colts’ ball nor the Patriots’ balls when tested by Anderson would have measured consistently with the pressures at which each team had set their footballs prior to delivery to the game officials, 13 and 12.5 psi respectively. Mr Wells’ testimony was confirmed by that of Caligiuri and Marlow. As Marlow testified, “There’s ample evidence that the non-logo gauge was used”.

[2] For similar reasons, I reject the arguments advanced in the AEI Report. The testimony provided by the Exponent witnesses and Professor Marlow demonstrated that none of the arguments presented in that report diminish or undermine the reliability of Exponent’s conclusions.

If Snyder’s testimony was as represented, he was a singularly poor choice of expert witness.  There are major errors, defects and adverse assumptions through the Exponent report and Snyder should have taken issue with them.  Why they wouldn’t have challenged the 67 deg F assumption of the simulations or the apparent gross error in Figures 26 and 30 (at CA here; also at ^ here)  is beyond me.

It’s also hard to understand why the Brady side would have produced an expert witness who hadn’t gone to the trouble of doing his own independent analysis.   As represented by Goodell, Snyder focused on the single issue of “timing”, claiming that Exponent had “ignored” timing.  While there are issues with how Exponent handled timing, it is ludicrous to say that they “ignored” timing issues.  Yes, their “statistical analysis” in Appendix A, from which ludicrous claims of “statistical analysis” are derived, ignored timing issues, but timing issues were front and center in the simulations and claims that Exponent “ignored” timing – if Snyder made such claims – are easily refuted.

On the other hand,  Goodell and Exponent’s characterization are always shadow boxing with reality.  Goodell said :” Dr Caligiuri and Dr Steffey both explained how timing was, in fact, taken into account in both their experimental and statistical analysis. ”  This isn’t true either. Timing was taken into account in their experimental analysis, but not in the statistical analysis (in Appendix A).   (By the way, I haven’t written except in passing about the statistical analysis in Appendix A as the simulations seemed to me to be the core of the prosecution case, while the statistical analysis was so stupidly irrelevant and pointless as to be worthless, but the fact that it is referred to here as a factor in the decision may cause me to revisit this.)

One of the most important, if not the most important, arguments in trying to make sense of events was the scenario in which referee Anderson used the Logo gauge for measuring Patriot balls and the Non-Logo gauge for measuring Colt balls, inattentively changing gauges between measurements – as NFL officials also did during half-time, despite the heightened scrutiny.  This scenario neatly reconciles a lot of otherwise discordant information, as discussed in previous posts.  This scenario was raised in the AEI article as well and in an early response by the Patriots.  If Goodell has correctly characterized evidence from Snyder and the NFLPA, they botched this issue as well. According to Goodell, they argued that Anderson had used the Logo gauge for measuring both Patriot and Colt balls, raising the problem of the approximate pregame match of Colt pressures and Anderson’s measurements.  This argument is moot if, as seems entirely possible, Anderson inattentively changed gauges.  Then the issue is how Anderson’s pregame measurement of Patriot balls (if done with Logo gauge) could have reconciled with Patriot pregame measurements.   On this narrower issue, there are a couple of possibilities: (1) the Patriot (Jastremski) gauge might have had a similar bias to Anderson’s Logo gauge.  Exponent’s analysis of gauge variation is wildly irrelevant to the problem as they limited their analysis to other examples of new Non-Logo gauges. Also, the NFL appears to have been in possession of the Jastremski gauge at half-time and could have tested its calibration, but it didn’t do so, apparently not keeping track of the gauge.  (2) while Exponent has plausibly shown that the additional pressure arising from Patriot gloving protocols would have worn off by the time of Anderson’s measurements, it also appears possible that Patriot pregame measurements were done while the balls were still impacted by gloving.

The AEI report had raised the issue of switching gauges, but did not carry out the more detailed analysis of the implications of that scenario on the transients and simulations.  The Brady side needed more than provided in the AEI report, but the switching scenario cannot be trivially dismissed either. Goodell stated: “The testimony provided by the Exponent witnesses and Professor Marlow demonstrated that none of the arguments presented in that report diminish or undermine the reliability of Exponent’s conclusions.”  I don’t see how anyone can responsibly assert that the switching scenario does not “diminish or undermine the reliability of Exponent’s conclusions.” It’s an important possibility that really does call into question the validity of Exponent’s claim that the decline in pressure cannot be accounted for by environmental and physical factors.

I noticed that Goodell’s decision added the word “substantial” in saying that “substantial part of the decline” was due to tampering. This word is new in the Goodell decision and is not actually stated in the Wells Report, which said instead that the decline “cannot be explained completely” by environmental and physical factors. As I reported previously, Exponent said that pressures in Exponent’s Game Day simulations were “noticeably higher” than observed Patriot pressures, but did not use the word “substantial” – undoubtedly because the difference was only 0.1-0.24 psi (see Figure 30 and Exponent page 62).

It seems to me that the use of the word “substantial” changes the hurdle. Would the difference of 0.1-0.24 psi, described as “noticeable” in the Exponent Report also be fairly described as “substantial”? I don’t think so. Read carefully, I do not believe that the Exponent Report, even on its own terms, supports the term “substantial” (as opposed to, say detectable).

Raymond Bradley and the Grand Old Duke of York

In today’s post, I’ll return to more typical Climate Audit programming.  Upside-down Mann’s mentor, Raymond Bradley, has somewhat surprisingly published an article (Balascio et al 2015) that supports a longstanding Climate Audit criticism of varve proxies. Bradley and coauthors did not report that their interpretation of an important Baffin Island series is upside-down to the orientation used in PAGES2K and numerous AR5 vintage multiproxy reconstructions.  It seems that proxies used by the Team are like the Grand Old Duke of York:

And when they were up, they were up,
And when they were down, they were down,
And when they were only half-way up,
They were neither up nor down.

Continue reading

Ruling out high deflation scenarios

Further to my series of posts on Deflategate, reader chrimony observed that my statistical analysis had shown that it was possible that there had been no tampering, but had not excluded the possibility of tampering.  This is a sensible observation, but raises the question of whether and how one could use the available statistical information to exclude tampering. This is analysis that ought to have been done in the Wells Report.  I’ve done the analysis in this post and the results are sharper than I’d anticipated.

For Logo initialization, any manual deflation exceeding de minimis of say 0.1 psi can be excluded by observations.  For Non-Logo initialization, statistical information rules out “high” deflation scenarios i.e.  deflation by more than the inter-gauge bias of 0.38 psi plus uncertainty, including deflation levels of ~0.76 psi reported in Exponent’s deflation simulations.  Remarkably, for Non-Logo initialization, the only manual deflation that is not precluded are amounts equal (within uncertainty) to the inter-gauge bias of ~0.38 psi.  Precisely why Patriots would have deflated balls by an amount almost exactly equal to the bias between referee Anderson’s gauges is a bizarre coincidence, to say the least.  I think that one can safely say that it is “more probable than not” that referee Anderson used the Logo gauge than that such an implausible coincidence.
Continue reading

Exponent’s Transients: Bodge or Botch?

In my first writeup, I observed that Exponent’s Logo transients appeared to be bodged too high, even with their unwarranted and adverse use of 67 deg F initialization (Exponent’s “temperature trick”). In today’s post, I’ve taken a closer look at the seemingly questionable calculation of the transients at 67 deg F, showing that the Patriot transients make sense only if initialization for the transients purporting to show Logo Gauge initialization were not actually initialized at 12.5 psi using the Logo Gauge (as stated and as is the purpose of the diagram).  My reverse engineering shows that the Patriot dry transient in Figure 27 only makes sense if the Logo Gauge read 12.81 psi at initialization or if the Master Gauge (not the stated Logo Gauge) was erroneously used for initialization.  If I’m correct, this is a very significant error –  a botch, rather than a bodge – for which one would expect a prompt corrigendum, if not retraction, of the corresponding calculations.  In a postscript to today’s post, I’ve attached a note on conversion from Logo and Non-Logo Gauge scale to correctly calibrated Master Gauge scale.

Continue reading

NFL Officials Over-Inflated Patriot Balls

One of the ironies of the NFL’s conduct in this affair is that it can be established that NFL officials (under the supervision of NFL Executive Vice President Troy Vincent) over-inflated Patriot balls at half-time, the only proven tampering with Patriot balls. Brady and the Patriots were unaffected by the overinflation by NFL officials, as they destroyed the Colts in the second half.

Exponent must have noticed the over-inflation by officials, as it is implied by the post-game measurements, but failed to report or comment on it. Their avoidance becomes all the more conspicuous because many of the texts at issue in the Wells Report pertain to an earlier incident in which NFL officials had over-inflated Patriot balls, much to Brady’s frustration and annoyance at the time.
Continue reading

More on Deflategate

By converting football pressures to ball temperatures under the Ideal Gas Law, it is possible to conveniently show Colt and Patriot information – transients, simulations and observations – on a common scale. I’ve done this in the diagram shown below, and, in my opinion, it neatly summarizes the actual information. Commentary follows the figure.

Figure 1.  Transients as digitized from Figures 25 and 27 converted to temperature transients using Ideal Gas Law. Red- Patriot, blue- Colt; thick – dry; thin – wet; solid -Logo, dashed – Non-Logo.  Simulations shown in open circles: large –  Logo 67 deg F initialization; small – NonLogo 71 deg F initialization.  Observed average: solid circle- Non-Logo; + – Logo. Continue reading

Deflategate and Errors in the Wells Report

Readers in the U.S. are doubtless aware of the “Deflategate scandal”, in which the NFL alleged that Tom Brady, the greatest quarterback of his generation, had conspired with an equipment manager and locker room attendant, to deflate a microscopic amount of pressure from footballs in the AFC championship game. The NFL seemed to be completely taken by surprise by the Ideal Gas Law and the fact that outside temperatures below calibration temperatures would result in much larger deflation without tampering.

The findings depend on the interpretation of statistical data by decision-makers – a topic that interests me.   I found the technical report by Exponent, Wells’ technical consultants, to be very unsatisfactory on numerous counts:

  • although they were reported by Wells to have considered “all permutations”, they hadn’t.  On important occasions, they omitted highly plausible possibilities that indicated no tampering and, on other occasions, they only considered assumptions that were most adverse to the Patriots;
  • on key occasions, it seemed to me that Exponent failed to properly characterize exculpatory results.

At the end of my analysis, I concluded that their key technical findings were simply incorrect and wrote up my analysis, now online here.

I watched both the AFC championship and the final. I have no fan commitment to the Patriots. As someone who’s played sports all his life and whose play has always been rushed, I am amazed at how time seems to stand still for great athletes such as Brady.

The summary is as follows.
Continue reading


Get every new post delivered to your Inbox.

Join 3,633 other followers