Thank goodness for diligent regulators. Read on, and be horrified.

The US Food and Drug Administration (FDA) recently inspected a testing facility outside Los Angeles and found serious concerns about the quality and integrity of the data produced there. In its warning letter, which is publicly posted online, the agency’s inspectors outlined the objectionable conditions they found.

Anyone interested in the role of this powerful agency in protecting the American public health should read this letter. Prepare for a glimpse into the “wild west” of preclinical research. Prepare to be appalled!

Background

Drugs and medical devices undergo extensive testing before touching human beings in clinical trials. These studies are called non-clinical (or preclinical) research. In the United States these studies are regulated under the Good Laboratory Practice regulations. As part of its Bioresearch Monitoring Program, the FDA inspects testing facilities to ensure the data generated is scientifically accurate and valid.

These inspections can result in a range of actions. For a well-run facility, management actually benefits from such an inspection. They get feedback on weak areas and they better understand the agency’s thinking on certain scientific or organizational issues they face.

For a poorly run facility, the inspection could result in a warning letter at best or an immediate shutdown of the facility, followed by legal prosecution, at worst.

This letter indicates the facility is poorly run. Appallingly so. As such, all the scientific data it has generated on the investigational drug is in question.

Some choice passages from this warning letter

There were no records indicating the presence of a functional QAU, or records of any QA activities at the facility. In addition, you indicated during the FDA inspection that you were the study director as well as the head of the QAU. However, this practice undermines the QAU from performing its required functions separate from and independent of the personnel engaged in the conduct of nonclinical laboratory studies. You also indicated that a veterinarian for the study from (b)(4), was a member of the QAU; however, that veterinarian signed a statement denying that he was a QAU member.

It’s going to be hard to walk that one back!

Considering your test facility management’s overall lack of responsibility to implement basic essential elements of GLP compliance, the quality and integrity of the study data cannot be assured because there was no QAU oversight of the nonclinical laboratory studies conducted at your testing facility.

This is pretty bad. The QAU must monitor the conduct and reporting of the organization’s studies. When it comes to life-impacting drugs and devices, the integrity and acumen of the researcher alone is not enough.

Your testing facility failed to establish standard operating procedures (SOPs) in writing setting forth nonclinical laboratory study methods that are adequate to insure [sic] the quality and integrity of the data generated in the course of the studies, as well as to ensure appropriate handling and care of animals.

During the inspection, you indicated that as a surgeon, you knew all operation procedures, and therefore you did not agree that SOPs needed to be established.

Written SOPs, with appropriate approvals and control, are a bedrock of quality testing. It’s hard to overstate this.

For a reader who does not have a scientific background, let me make an analogy: Have you ever been in a court of law? What if you were on trial for a capital crime, and your lawyer showed up with no computer and no documents. You ask him, “What’s up, did you not prepare at all?” In response, he taps his temple and grins, saying, “Don’t worry, it’s all up here!”

I am impressed with the incredible capabilities of the scientists that I work with every day. But they and I know the importance of independent checks. The surgeon the FDA interviewed is saying his authority and expertise puts him above independent checks. He is saying we should trust his data because of his skill and expertise. Sorry, but this is not a medieval guild. Mastery is not a validated method, and modern drugs and devices require independent examination and comparison with a standard.

As the study director, you must ensure that all experimental data, including observations of unanticipated responses of the test system, are accurately recorded and verified.

You failed to ensure that all experimental data were accurately recorded and verified.

For the toxicity study in dogs, dose calculation worksheets for nine dogs show body weight measurements and test article administration in June and July 2016; however, purchase records indicate that eight of the nine dogs were purchased in August 2016, which makes the data entered before August 2016 invalid. In fact, during the FDA inspection, you acknowledged that the data entered before the dosing in August was inaccurate and did not reflect real-time data entry.

For all data changes, you failed to ensure: (1) that the original entries were not obscured; (2) that the reason for all changes was indicated; and (3) that all changes were dated and signed or identified at the time of the changes.

The study director’s role in a preclinical study is extremely important. He or she serves as the single point of control for the study. The passages above speak for themselves. They may even point to falsified data, though this is not explicitly suggested in the warning letter.

Basic data integrity principles and methods are taught to undergraduate science students within the first week of their laboratory courses. These include correcting handwritten entries without obscuring the original entry, signing and dating, documenting concurrently, et cetera. The warning letter shows these principles were not followed at this facility.

Because you failed to ensure that all experimental data were accurately recorded and verified, and that all entries and changes in entries were properly documented, FDA has concerns about the integrity of the data generated from the nonclinical toxicity studies conducted at your testing facility.

The investigators use professional-sounding understatement.

You failed to comply with the above requirements to retain and to provide archives for storage and retrieval of all raw data and specimens.

You were unable to provide all data requested during the inspection; you indicated that you were unsure where the data was, and that it might be at your home.

The absence of raw data and specimens collected during these nonclinical toxicity studies, and the absence of proper storage of raw data and specimens, raises significant concerns about the integrity of the study records and data.

Again, understatement. Archival procedures, controlled storage conditions, data completeness, and traceability are a bedrock of quality. These procedures would be in the SOPs, but as noted above, the facility’s SOPs were inadequate or absent.

Testing to determine the stability of the test and control articles in the mixture must be conducted, either before the study initiation or concomitantly, according to written SOPs that provide for periodic analysis of the test and control articles in the mixture. However, none of these analyses were done.

Does this sound a bit esoteric to you? It’s not. Before testing a compound in preclinical toxicity trials, the compound must be well-characterized. This ensures that the compound that eventually is dosed in humans matches the compound that was given to animals or to in vitro test systems. Without this characterization, you are putting humans in those safety trials at risk, or generating poor-quality data.

The protocols of nonclinical toxicity studies in mice, hamsters, rabbits, and dogs… did not have a date of approval by the sponsor and did not contain sufficient details, including but not limited to:

a. A statement of the purpose of the specific study

b. The name and address of the testing facility at which the study is being conducted

c. The procedure for identification of the test system

d. The methods for the control of bias

e. The frequency of tests, analyses, and measurements to be made

f. The records to be maintained

The study protocol is a central document. To name just two possible consequences of not having one: How do you identify a deviation from the study plan, if no plan was documented? How do you evaluate the outcome, if no predetermined endpoints were established? Not having an adequate protocol leads to weak scientific data and conclusions.

As evidenced by FDA’s inspection findings, your testing facility management failed to fulfill the primary responsibilities to establish appropriate policies and procedures intended to ensure the quality and integrity of nonclinical data for FDA submission. Furthermore, the deficiencies found in your oversight as the study director and the absence of an independent QAU indicate that your testing facility failed to fulfill the core responsibilities to remain GLP-compliant. As a result, FDA is concerned about the validity of nonclinical data generated by your testing facility.

Failure to address the violations noted above adequately and promptly may result in regulatory action without further notice.

More understatement. I haven’t checked, but this facility may already have been shut down! The FDA has the power to walk in and immediately halt operations. Especially since this is not their first warning letter.

Some caveats

I don’t wish to pile on. I don’t want to be part of an Internet mob. And I always assume good intent instead of nefarious fabrication and fraud. But people should be aware of the risks to health out there. The FDA cannot inspect every facility all the time. They cannot catch all the shoddy science out there. They can’t prevent people from wasting money on ineffective treatments.

I also feel sorry for the animals in these studies, which include mice, hamsters, rabbits and dogs. The letter indicates there were no SOPs describing their care and handling. What if some of them suffered from poor treatment? What if their treatment was fine, but the data from their studies was completely invalidated? It’s as if they underwent testing for nothing. If an animal was used in research, something good should come out of it. This may not have happened here.

I will write an article about how to use FDA warning letters and other materials to be proactive in your organization

In a future article I will show you how to use the FDA’s warning letter database to your advantage. The point is not just to gawk at the egregious ones. You can use the warning letters to get an edge and understand the agency’s thinking on many biotech and pharmaceutical subjects.

What do I mean by “getting an edge?” Let me explain. Have you noticed how regulations change very slowly? The GLP regulations that we follow were finalized in the 1970s and 1980s! There is nothing in those regulations mentioning cloud-based data capture, gene editing, or artificial organs. The FDA occasionally releases non-binding guidance documents that help industry, with its cutting-edge techniques, from falling afoul of these older regulations. But even those guidelines quickly become dated and cumbersome.

FDA warning letters, on the other hand, reveal findings and enforcement actions from inspections that occurred only weeks or months previously. The letters also indicate the reasoning behind the actions. A useful FDA inspection finding includes three things: a detailed description of the circumstance, the applicable standard (usually a regulation), and the possible consequences of not meeting the standard.

Read closely and you will find all three components in the letter I linked above. In my upcoming article I will break down the warning letters in a way that will make you a better auditor who will be even more valued by your organization or auditee.

Final comments

Visit the FDA warning letter database regularly. Stay tuned to warning letter news. Read my upcoming article on warning letters. Let me know if this article benefited you. And let me know if your jaw dropped as much as mine did!

Image credit

The image is from http://www.cancertreatmentus.org/