As a starting point for today’s post, the Office of Research Integrity (U.S. Department of Health and Human Services), like Penn State, has a two-stage inquiry-investigation procedure and states:
In general, absent full admissions, inquiries should not be used to make findings on whether research misconduct in fact occurred.
This seems at odds with what the Penn State Inquiry Committee has done or is perceived to have done. Today I’ll try to examine the difference between an “inquiry” and an “investigation” in academic misconduct cases and see what the Mann Inquiry Committee did or thinks that it did.
Penn State Policy RA-10
Penn State policy RA-10 refers to both “inquiry” and “investigation” stages – a structure consistent with the other U.S. federal academic misconduct procedures. The distinction between the two stages is spelled out in much more detail by the Office of Research Integrity, but it’s clear that Penn State RA-10 had adopted the same two-stage structure as described in more detail elsewhere.
RA-10 section 4 describes the obligations of the inquiry committee if it decides “an investigation was not warranted” – it is obliged to provide a record containing “documentation in sufficient detail to permit a later assessment, if necessary, of the reasons for determining that an investigation was not warranted”, which is to be retained for up to three years and made available upon request to any involved Federal agency. RA-10 section 5 describes the procedures if the inquiry determines that the allegations “warrant further investigation”, i.e. the formation of an investigation committee which is obliged to investigate the matter.
ORI Description of the Two-Stage Procedure
ORI here provides far more detailed definition of the two-stage procedure. They state:
The existing final rule defines a two-stage process that takes place when an institution receives allegations of research misconduct:
(1) An inquiry, or preliminary fact-finding, to determine if the allegation involves PHS supported research and has sufficient substance to warrant an investigation; and
(2) an investigation, which is a thorough review and analysis of all relevant facts to reach a conclusion as to whether research misconduct has occurred, who was responsible, and how serious any misconduct was.
Institutions treat the inquiry phase in a widely varying manner, and the distinction between an inquiry and an investigation has caused much confusion. Some inquiries conducted by institutions are largely indistinguishable from investigations. As the OSTP policy adopts a two-stage process, we have retained the current two-stage process but propose to sharpen the distinction between inquiries and investigations by clarifying that the inquiry is only an initial review of the allegations to see if they warrant an investigation.
An “inquiry” (Sec. 93.213) is defined as follows:
Inquiry means preliminary information-gathering and preliminary fact-finding to determine whether an allegation or apparent instance of research misconduct has substance and if an investigation is warranted.
While an investigation (Sec. 93.216) is:
Investigation means the formal development of a factual record and the examination of that record leading to a decision not to make a finding of research misconduct or to a recommendation for a finding of research misconduct or other appropriate remedies, including
Sec. 93.307(a) sets out criteria warranting an inquiry if the allegation:
(1) Falls within the definition of research misconduct under this part;
(2) Involves PHS supported research; and
(3) Is sufficiently credible and specific so that potential evidence of research misconduct may be identified.
Section 93.307(c-d) describes the purpose of an inquiry as the determination of whether to conduct an investigation:
(c) Purpose of an Inquiry. The purpose of an inquiry is to conduct an initial review of the evidence to determine whether to conduct an investigation. Therefore, an inquiry does not require a full review of all the evidence related to the allegation.
(d) Criteria warranting an investigation. An inquiry’s purpose is to decide if an allegation warrants an investigation. An investigation is warranted if there is–
(1) A reasonable basis for concluding that the allegation involves PHS supported research and falls within the PHS definition of research misconduct; and
(2) Preliminary information-gathering and preliminary fact-finding
from the inquiry indicates that the allegation may have substance.
Sec. 93.105 prescribes time limitations. (I don’t know specifics of the Penn State regime but they might be similar):
(a) Six-year limitation. This part applies only to research misconduct occurring within six years of the date HHS or an institution
receives an allegation of research misconduct.
(b) Exceptions to the six-year limitation. Paragraph (a) of this section does not apply in the following instances:
(1) Subsequent use exception. The respondent continues or renews
any incident of alleged research misconduct that occurred before the six-year limitation through the citation, republication or other use for the potential benefit of the respondent of the research record that is alleged to have been fabricated, falsified, or plagiarized.
ORI has a sort of ‘interpretation bulletin’ (borrowing a phrase familiar to lawyers) on the distinction, which I’ve reproduced in full below, as it is relevant to the present situation:
(21) Inquiry Versus Investigation Stages in Research Misconduct Cases – When examining allegations of research misconduct, institutions have an obligation to conduct an initial inquiry, and, if warranted, a thorough investigation in accordance with PHS Policies on Research Misconduct. 42 C.F.R. Part 93. This article discusses some procedural and substantive considerations in examining allegations of research misconduct in inquiries and investigations.
After receiving a good faith allegation of research misconduct, an institution will usually open an inquiry to gather general information and make initial findings of fact to determine whether the allegation has substance and there is sufficient evidence to warrant an investigation. Sometimes, however, when there is sufficient evidence already at hand, for example as the result of an audit of a clinical trial, the institution may move directly to the investigation stage. If the inquiry uncovers evidence of “fabrication, falsification or plagiarism,” an investigation is warranted. Section 93.307 (d). In general, absent full admissions, inquiries should not be used to make findings on whether research misconduct in fact occurred.
On occasion, ORI receives an inquiry report in which either the committee has conducted the equivalent of an investigation and made specific findings, or which is obviously the result of a negotiated agreement. These reports may violate the PHS regulation and cause substantial difficulties for ORI’s oversight. Findings made at the inquiry stage are all too frequently incomplete because the record has not been fully developed, and negotiated agreements violate the PHS regulation, if made without advance notice to ORI. Both instances may deprive ORI of the facts necessary to determine whether there has been an adverse effect on the PHS sponsored research, and the institution may be required to reopen its case and initiate an investigation.
Instead of short circuiting the process, once an institution has determined that there is some evidence of possible misconduct, a thorough investigation should be conducted in accordance with the requirements explained in Section 93.310. Only after this process is complete should the investigation committee turn to an analysis of whether the charges have been proven by a preponderance of the evidence.
Even if the record is relatively complete at the inquiry stage, the PHS regulation gives respondents the opportunity to have a full investigation unless they admit misconduct and waive the investigation. This multiple stage process has also been endorsed by the Federal Office of Science and Technology Policy which has stated that the investigation is “the formal examination and evaluation of the relevant facts leading either to dismissal of the case or a recommendation for a finding of research misconduct or other appropriate remedies.” Federal Policy on Research Misconduct, 65 Fed. Reg. 76260-76263 (December 6, 2000).
The difference in function between an “inquiry” and an “investigation” is quite clear in this exposition: the purpose of an inquiry is, in effect, to determine whether there is any credible evidence supporting the allegations and whether the allegations, if true, constitute an offence. As I read the ORI procedure, defences and justifications are only relevant at the investigation stage and should not be entered or considered at the inquiry stage, when a full investigation has not taken place. (This is hopelessly jumbled in the Penn State Inquiry.)
ORI has an interesting report auditing inquiry reports for compliance with ORI policies – finding that the majority wre seriously deficient in one aspect or another:
ORI conducted a content analysis of 21 inquiry reports that were not submitted to the Office of Research Integrity (ORI) because an investigation was not recommended and ORI had not previously requested the report. The study addressed the following questions: (1) Were the inquiries being reported by institutions on the Annual Report subject to PHS jurisdiction? (2) Did the institutions sufficiently document the rationale for deciding that an investigation was unwarranted? (3) Did the conduct of the inquiries comply with the regulation? (4) Is more technical assistance needed in the conduct of inquiries?
This study demonstrated that more than half of the institutional inquiry reports that were not submitted to ORI were significantly deficient. Fifty-seven percent of the reports did not contain the information required to establish PHS jurisdiction. Thirty-three percent contained information on no more than four of the nine criteria used to determine whether an investigation was warranted and another 28 percent were marginal, covering only five criteria. Seventy-one percent provided information on only three or fewer criteria for determining compliance with the regulation. And finally, 57 percent of the reports did not contain the detailed information required to justify the decision that an investigation is unwarranted. The analysis was based solely on the content of the submitted reports. Additional information supporting the decision that an investigation was unwarranted may exist in other documents that were not submitted.
The Mann Inquiry Report is a rather odd document, when assessed against ORI procedures.
Obvious issues include (1) whether the “synthetic” allegations presented to the inquiry were sufficiently specific; (2) whether an investigation committee rather than an inquiry committee should have heard Mann’s defences to allegations 1-3.
Questions concerning the inquiry committee go the other way as well. Offhand, I can think of two extremely scoping issues that, in my opinion, should have been resolved at a scoping (inquiry) stage, rather than an investigation stage: (1) is there a statute of limitations on misconduct offences? (2) is conduct at prior employment subject to Penn State jurisdiction? In my opinion, Mann and his lawyer should have insisted on these questions being squarely addressed by the inquiry; they weren’t.
For example, regardless of what thinks of the “trick” – and I yield to no one in criticizing it – it happened 10 years ago at another university. Is it subject to a statute of limitations along the lines of the one referred to above? Does the fact that it happened at another university matter?
The handling (or lack of handling) of these questions by the Inquiry is totally incoherent. For example, they produced the following remarkable pastiche in connection with the “trick”:
While a perception has been created in the weeks after the CRU emails were made public that Dr. Mann has engaged in the suppression or falsification of data, there is no credible evidence that he ever did so, and certainly not while at Penn State. In fact to the contrary, in instances that have been focused upon by some as indicating falsification of data, for example in the use of a “trick” to manipulate the data, this is explained as a discussion among Dr. Jones and others including Dr. Mann about how best to put together a graph for a World Meteorological Organization (WMO) report. They were not falsifying data; they were trying to construct an understandable graph for those who were not experts in the field. The so-called “trick”1 was nothing more than a statistical method used to bring two or more different kinds of data sets together in a legitimate fashion by a technique that has been reviewed by a broad array of peers in the field.
In my opinion, the Inquiry Committee could have said – whatever the rights or wrong of the trick email, it’s past the time of limitations (IF there is an applicable statute of limitations). Or they could have said that it happened at another university (IF that’s the regulation).
However, it’s not open to them to entertain a positive defence (the excuse that the trick is a “good way” of doing something) to an alleged offence that took place ten years ago at another university and then passim use the phrase “let alone Penn State” in their “decision”. (Needless to say, if, as the Office of Research Integrity stated, “inquiries should not be used to make findings on whether research misconduct in fact occurred”, then it was premature of them to purport to make a decision anyway.)
Even though they purported to deal with three allegations, Allegation 4 remains remains on the table. But the Inquiry is silent on whether this allegation is limited to his conduct at Penn State or to his conduct within the last six years. Do the terms of reference for the investigation committee include Mann’s conduct since the beginning of time? This hardly seems fair.
Instead of trying to decide on things that were outside its terms of reference, the Mann Inquiry Committee should have provided decisions on the topics that were within its terms of reference. The Mann Inquiry Committee has probably made about as many procedural errors as it’s possible to make in the compass of a short report – and, as noted above, not all to Mann’s benefit.
Given the publicity that this case has attracted and will continue to attract, you’d have that the university would have obtained the best possible professional advice on how to handle an inquiry and investigation from the Office of Research Integrity (or their equivalent at NSF). Instead, like Gerry North, they seem to have decided to “wing it” – an unfortunate decision that will do nothing except prolong the melodrama.