Remote Evaluations, Grading Industry on a Curve and Using AI to Predict Site Inspection Scores
By: Axendia Staff
The FDA issued its latest report on the State of Pharmaceutical Quality. In the report, the Agency describes Pharmaceutical Quality as the ability of the pharmaceutical manufacturing industry to deliver quality drug products to U.S. patients and consumers. In addition, FDA defines quality drug products as those that are safe and effective and free of contamination and defects. It is worth noting the absence of the word “compliant” in these definitions.
Hidden among Hand Sanitizer recall data were a number of significant developments on FDA’s regulatory approach.
This Brief represents Axendia’s analysis and insights on FDA’s Report on the State of Pharmaceutical Quality.
Remote Evaluations are Here to Stay
During the global public health emergency, many onsite FDA inspections were not possible. Instead, the agency relied on Intelligence Based E-Inspections. In 2012, the Food and Drug Administration Safety and Innovation Act gave FDA new authorities under the Food, Drug, and Cosmetic Act §704(a)(4) to request records or other information from firms in advance of or in lieu of an inspection.
Although the number of surveillance inspections conducted significantly dropped in March 2020, FDA used this authority to obtain records for sites that would have had surveillance inspections. (See Intelligence Based E-Inspections )
FDA also expanded its reliance on Mutual Recognition Agreements (MRAs) with the European Union and the United Kingdom to include inspections performed in countries other than their own. (See FDA Wants to Transform into a Global Agency )
In addition, the FDA used several innovative strategies to assess and act on regulatory submissions that would typically have needed a pre-approval inspection (PAI) or pre-license inspection (PLI). The agency used surveillance history, requests for records, and inspection reports obtained through the MRAs to mitigate risk and enable regulatory actions. In fact, these approaches reduced the need to conduct PAIs 58% of the time in FY2020 Q3 and resulted in 183 site assessments through MRA inspection reports.
FDA is Grading Industry on a Curve
FDA uses site inspection scores (SIS), on a scale of 1 to 10, as a proxy for compliance with CGMP regulations. Compliance with CGMP assures proper design, monitoring, and control of manufacturing processes and facilities, and represents the minimum standards to which sites must adhere. Higher SIS scores indicate better compliance with CGMP. (See FDA to Grade Industry on a Curve – Are You Ready? )
The SIS is based on the classification of FDA drug quality assurance inspections conducted over the prior ten years, including inspections classified under the MRA program, which allows some global regulators to recognize reports from their counterparts’ inspections. As sites are inspected and receive a classification outcome, these outcomes are transformed into an SIS. ( See: Is Your Quality Unit Ready to Submit Quality Metrics Data Across Your Supply Chain? )
AI to Predict Site Inspection Scores
FDA is using eXtreme Gradient Boosting machine learning regression to model and predict site inspection scores. The AI uses inspection data from 10 years of inspection outcomes. Leveraging AI, the Agency has identified a number of features and feature combinations that can predict an increase or decrease in the risk factor associated with low or high SIS. The features considered included Profile Class Codes (PCCs). When a drug manufacturing inspection is performed, FDA uses PCCs to capture all product classes produced by the manufacturer as well as those covered during the inspection (e.g., tablets, capsules, ointments, sterile injectables).
The number of application and non-application products being manufactured at a site were the top two most important features associated with higher or lower SIS, respectively. These types of analyses enable FDA to better target and apply surveillance resources.
Measuring Your Commitment to Quality
During FY2020, while tackling the demands of the COVID–19 public health emergency, FDA responded by introducing new, and adapting existing, surveillance tools to watch over the drug supply chain while non-mission-critical inspections were postponed due to travel restrictions. Through proactive efforts CDER seeks to improve the future state of pharmaceutical quality and to minimize long-standing problems such as drug shortages due to quality issues.
These efforts include:
New Inspection Protocol Project (NIPP): This project is aimed at using standardized electronic inspection protocols to collect data in a structured manner. The protocols promote consistent and comprehensive coverage of critical areas of drug manufacturing and provide structured, data-rich reports. The protocols include questions related to quality culture observed in facilities. In the future, FDA will have the ability to better understand how certain variables (e.g., location of the establishment, type of establishment) affect quality. As more data are collected through NIPP, these types of insights can inform future inspections, identify policy/outreach opportunities, and influence application-related decision making. (See: Is Your Quality Unit Ready to Submit Quality Metrics Data Across Your Supply Chain? )
Quality Management Maturity: To characterize quality management maturity among drug manufacturers, FDA funded research to study over 200 pharmaceutical manufacturing establishments. The researchers found that quality has a positive correlation with performance, as there was a high degree of positive correlation between:
- Delivery indicators such as order fulfillment, customer complaint rate, and adherence to standard lead time
- Measures of the degree to which an organization adheres to best-practice maturity principles
These findings support the hypothesis that a high degree of quality management maturity has a positive impact across an organization, including on the fundamental ability to deliver high quality drug products. (See: Show FDA Your Commitment to Quality )
Regardless of the circumstances, U.S. patients and consumers deserve confidence in their next dose of medicine. (See FDA: Consumers and Patients Deserve Six Sigma Quality Products )
FDA continues to engage with stakeholders, respond to changing circumstances based on science and risk, develop innovative programs, and share information on the State of Pharmaceutical Quality, to assure the availability of safe, effective, quality medicines.
Based on Axendia’s interactions with the Agency it is clear that:
Remote Evaluations Are Here To Stay: According to our research, 50% or organizations are ready for them (See: Technology’s Role in Overcoming a Disrupted Life-Sciences Reality ). Yet, FDA is making increasing use of FDA’s authority under §704(a)(4) of the FD&C Act to request records from regulated firms. Furthermore, FDA recently published guidance on Remote Interactive Evaluations to enhance surveillance oversight while decreasing risks of exposure during a pandemic. These tools have allowed many products to be approved, without PAIs or PLIs, before their User Fee dates and prevented many inferior products from entering the U.S. market.
Quality Metrics Are Key in The New Normal: FDA is moving toward a rating system that incentivizes drug manufacturers to invest in quality and achieve higher levels of quality management maturity. The New Inspection Protocol Project aims to use standardized electronic inspection protocols to collect data in a structured manner.
Documented Evidence Is Not Sufficient: FDAis transitioning from documented evidence to Digital Evidence to support Digital Transformation. (See FDA Discusses the Use of Digital Evidence to Accelerate Innovation )
Contact Research@axendia.com to schedule an Analyst Inquiry on this topic.
The opinions and analysis expressed in this Briefing Note reflect the judgment of Axendia at the time of publication and are subject to change without notice. Information contained in this document is current as of publication date. Information cited is not warranted by Axendia but has been obtained through a valid research methodology. This document is not intended to endorse any company or product and should not be attributed as such.