QuarterWatch™ (Special Report) A critique of FDA’s key drug safety reporting system
A new, special edition of ISMP’s QuarterWatch™ closely examines the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS), which combines mandatory reports originated by drug manufacturers with voluntary cases submitted by health professionals and consumers directly through FDA’s MedWatch program. For the past 6 years, this reporting system has formed the core data source of ISMP’s QuarterWatch™ reports about drug safety. But the FDA reporting system is in need of modernization to address the changing environment in which drugs are marketed. It also suffers from a flood of low quality reports from drug manufacturers who are required to submit reports of all adverse events and deaths about which they learn. We discuss the key problems below and offer recommendations as an organization that relies heavily on data collected through FAERS.
Volume of reports. This report examines 12 months of data ending with March 31, 2014. During the study period, the FAERS received 847,039 reports, including 41,884 deaths outside the US and 45,688 deaths in the US. As explained in detail below, the large number of deaths includes cases in which a patient died but the drug’s contribution to death was not investigated. The analysis includes reports of injuries that were not classified as serious, cases from drug company legal departments, and foreign reports, which are all typically excluded in the QuarterWatch™ analysis.
Entities submitting reports. During the 12-month study period, 96% of the adverse event reports were from manufacturers, and only 4% of the reports were submitted directly to FDA’s MedWatch program. Health professionals and consumers can report adverse events directly to FDA or to manufacturers. Thus, the overall quality and value of drug safety surveillance in the US depends primarily on how well drug manufacturers collect, code, and follow up on adverse drug events of which they become aware.
Key Problems Identified with FAERS
Incomplete manufacturer reports. For the 12-month study period, 85% of serious reports submitted by health professionals and consumers directly to FDA were reasonably complete, meaning they contained the patient’s age, gender, and event date, all factors that could be important during analysis. By contrast, only 46% of serious reports submitted by drug manufacturers were reasonably complete. Even with expedited reports, which are required within 15 days of learning about any unexpected (not in prescribing information) event with a serious outcome, only 49% were reasonably complete. While reasonably complete reports improved to 62% upon revision of initial expedited reports, manufacturers did not reach 85% of reasonably complete reports as seen with direct reports to FDA. Among cases indicating some of the most dire health outcomes, even fewer reports were considered reasonably complete—only 25% of birth defect cases, for example. We concluded that manufacturers, overall, did a poor job of collecting basic patient and event data, regardless of the severity of the event.
We also analyzed the widely varying performance of individual manufacturers that submitted at least 500 reports in the 12-month study period. The weakest performance was seen with five companies that submitted reasonably complete reports in 16% or fewer cases—Par, Cubist, Roxane, Millennium, and West-Ward. The five strongest performers submitting reasonably complete reports in 72% or more cases were Ariad, GE Healthcare, Vertex, United Therapeutics, and Biogen Idec. These data show that the burden of creating even thousands of reports is not necessarily an obstacle to higher quality reporting.
Promptness of reports and label changes. Identifying new safety issues quickly is an objective of FAERS and distinguishes it from other government safety and mortality assessments that often provide quality data but years after the fact. In the 12-month study period, FAERS received most new cases in a timely manner. For example, manufacturers submitted 88.5% of the initial expedited reports within the required 15 days. However, this does not guarantee that reports are rapidly translated into new restrictions and warnings. Once a signal is clearly identified, a label change can often take a year or longer and involve extensive interactions with the manufacturer.
Inference of causality. The central concept underlying adverse drug event reporting is an inference—but not a certainty—that the suspect drug caused the adverse event. As the case count grows describing an adverse effect, the inference of causality is strengthened. However, QuarterWatch™ has observed an increase in the number of cases in FAERS where the report did not reflect a suspected adverse effect of a drug, but instead was a byproduct of various programs in which the drug manufacturer initiated contact with patients or health professionals. This includes contact through prescription refill reminders, assistance with insurance coverage, nurse hotlines, and patient education programs.
In the process of interacting with patients or health professionals, companies may learn of a death or other adverse outcome. For example, the manufacturer of a biological product to reduce the risk of respiratory syncytial virus (RSV) in premature infants telephoned parents of children who received this product every month to remind them to get monthly injections for their infants during the infectious disease season. During these calls, the manufacturer learned of the deaths of many premature infants and therefore reported them to FAERS although the product may not have been the direct cause of an infant’s death. To cite another example, Biogen Idec told us that each quarter it telephoned every multiple sclerosis (MS) patient taking its drug interferon beta-1a (AVONEX). In the process, it learned of and reported many MS relapses, which are expected in a relapsing/remitting disorder. In these cases, the link between the drug and relapse is doubtful, and such cases cannot be easily separated from those where the drug was thought to have exacerbated MS.
Patient and health professional interactions that trigger adverse event reports also occur because of high-risk drugs with restricted distribution programs (e.g., sodium oxybate, natalizumab) or other Risk Evaluation and Mitigation Strategy (REMS) programs required by FDA. Regular interactions may trigger adverse event reports at much higher rates than for drugs without programs that involve extensive interaction. In other cases, when patients with life-threatening diseases such as metastatic cancer or pulmonary arterial hypertension interact with a drug manufacturer through a patient support program, the company will frequently learn the patient has died. By FDA regulation, this death must be reported regardless of whether the drug was suspected of contributing to the patient’s death.
For patient deaths during the 12-month study period, 28.5% (n = 24,939) of the reported deaths provided no information about whether the drug was suspected of contributing to the fatality. In thousands of other cases, the reported adverse event was the patient’s underlying disease without a determination or suspicion that the drug therapy had exacerbated it. Examining all of the cases, we found that 67% of reported patient deaths were of limited value, with only the single event term “death” reported, or with other information missing that would help assess the event.
In addition, manufacturer contact with consumers can lead to companies learning about minor everyday health problems—for example, injection site pain, a common cold, or a stuffy nose. These cases added little or no value to drug safety assessment. The FAERS was designed to receive spontaneous reports about suspected drug injury from manufacturers, health professionals, and consumers. The FDA regulations that require manufacturers to report these events were written before the modern era during which drug companies often seek to interact directly with as many consumers as possible.
Gaps in system coverage. Based on our analysis, FAERS works best in postmarket surveillance for adult patients taking newer brand name drugs, and is weakest for newborn infants, children, and patients taking older generic drugs. Both FDA and QuarterWatch™ have previously reported that the number of reports in children under age 18 were too few for effective postmarket surveillance. While the age group accounts for 24% of the population and 7.3% of the dispensed outpatient prescriptions, only 3% of the adverse event reports involved children during our study year. For birth defects, reporting is even more limited (n = 3,248), compounded by poor quality reporting. The Centers for Disease Control and Prevention (CDC) estimates 3% of approximately 4 million live births every year include a birth defect. We know of no credible estimates of drug-related birth defects, but cannot rule out that they number in the thousands. The current FAERS fails to improve our understanding of drug-related birth defects.
About 86% of outpatient prescriptions are for generic drugs, and similar requirements for reporting adverse drug events apply to brand and generic drug manufacturers. However, the major brand name manufacturers provide the overwhelming majority of adverse event reports, even when brand name drug sales are extremely small. For example, Pfizer accounted for 66.5% of all serious US adverse event reports for the antidepressant sertraline (ZOLOFT), even though its brand name product accounted for less than 1% of sertraline prescriptions in 2013.
Outdated FDA regulations. How drug manufacturers collect and report adverse drug events is specified in legally binding FDA regulations, described in substantial detail in guidance documents, and enforced through compliance inspections. However, the FDA guidance document is a draft that has never been approved and has not been revised since 2001. But since 2001, FDA has approved a growing number of drugs available with restricted distribution schemes or REMS programs that require intensive manufacturer contact with both patients and prescribing doctors, potentially spurring large volumes of reports. Also, manufacturers have increasingly implemented nurse-operated hotlines, provided assistance for applying for insurance reimbursement, and provided online and personal instructions for using certain products.
The growing problems with FAERS outlined above need to be addressed by FDA, which establishes and enforces reporting requirements, and by the industry, which must fulfill them. Three priority areas include quality improvement, manufacturer-initiated contacts, and monitoring of generic drugs.
Quality improvement. When a majority of manufacturer reports are not reasonably complete, quality improvement should be a priority. Given that reports from some manufacturers are much better than average and direct reports to FDA are at least 85% reasonably complete, the objective is possible. A simple computer-driven assessment for each manufacturer’s submissions could provide more extensive feedback than rare on-site company inspections by FDA. Birth defects and adverse events in children should be made a priority for increasing reporting rates and improving the quality of the reports.
Manufacturer-initiated contacts. The digital tools and marketing practices that now enable extensive contacts between manufacturers, health professionals, and consumers can be extended to provide enhanced postmarket surveillance. With a series of preapproved protocols for manufacturer-initiated patient or health professional contacts, it is possible to conduct enhanced safety surveillance and collect higher quality data. But these data are different from spontaneous reports and need protocols for the transactions, possibly new report data elements, specific lists of questions to ask, and clear identification of the reports obtained through these channels. Much could be gained from requiring a simple critical question in company-initiated contacts that asked: “Was the drug suspected of contributing to the event?” There is no point in manufacturers submitting thousands of reports of deaths in which a possible drug role was not alleged, ascertained, or investigated.
Monitoring generic drugs. A gap in monitoring generic drugs is not easily solved. Programs to publicize and expand FDA’s direct, online reporting system could be one answer. Another possibility is selecting health practices to be sentinels for generic drug adverse event reporting, like the CDC’s sentinel system for monitoring during the flu season.
Strengths of the FAERS System
For all its defects, the current FAERS has many vital features not seen in alternative drug safety surveillance programs. Since 2001, reporting has been standardized on a global basis, with a common terminology for describing adverse events and detailed definitions of what constitutes an event. Since 2014, an all-electronic system has been in place. With coverage of hundreds of millions of patients around the world, it successfully identifies rare but catastrophic side effects not clearly evident in pre-approval clinical trials. Statistical tools have been developed to identify signals in large data sets that were not collected systematically. The FDA has modernized its computer infrastructure, solved internal data quality problems, and developed a consumer-friendly online reporting form for direct reports. The system also captures reported events rapidly. However, it is possible to achieve substantial improvements in postmarket surveillance and better data quality with the modest and low cost improvements suggested above, and at the same time reduce the burden on drug manufacturers or focus existing resources in a more productive direction.
The full Quarterwatch™ report with references can be found here.