QuarterWatch™ monitors the safety of prescription drugs through analysis of adverse drug events reported to the U.S. Food and Drug Administration by consumers and health professionals, either directly to the agency or through drug manufacturers. The agency releases computer excerpts for research use on a quarterly basis, and these case reports are our primary data source.1 The agency data system for managing these reports is called FAERS (or the FDA adverse event reporting system). However, the agency promotes reporting of adverse drug events from drugs, medical devices and vaccinations under the program name “MedWatch,” which appears at the top of on-line and paper report forms.
This document provides detailed information about our methods, event definitions, and other procedures that support our findings. In addition, it is can serve as a guide to researchers who seek to conduct their own studies using FAERS data. Our current standard methods are outlined below. However, consult the individual QuarterWatch™ report to check whether special methods were required to address a particular safety issue.
Defining Serious Adverse Drug Events
Consistent with FDA regulation,2 an adverse event was classified as serious if the case report specified an outcome of death, disability, hospitalization, required intervention to prevent harm, was life threatening, or had other medically serious consequences. Cases without these serious outcomes were classified as not serious. In a policy change in 2016, all new cases were included in the basic analysis. Previous QuarterWatch™issues have focused primarily on a subset of adverse events, those that are domestic and coded with serious outcomes. Thus, reports released in 2016 and later include cases from foreign countries and domestic cases without a coded serious outcome. In the FAERS data release, however, a case report may indicate several different serious outcomes in the same case (e.g. hospitalization and death). In QuarterWatch™ reports serious outcomes are coded as mutually exclusive categories in this priority order: death > disability > life threatening > required intervention to prevent harm > hospitalization > other medically serious.
Classification of Drug Names
We define for analysis a “drug” as an active, unique chemical moiety that is FDA approved for the diagnosis or treatment of any disorder or disease. Most drug names are standardized to an ingredient name based on the National Library of Medicine drug name terminology called RxNorm. This means that salts, esters, extended release formations, and different routes of administration still appear under the same drug/ingredient name in QuarterWatch™ analysis. In most cases this makes little or no difference (e.g. amoxicillin sodium or amoxicillin trihydrate) but could have an effect on the analysis in special cases. For example, the immunosuppressant tacrolimus is available as an oral capsule to prevent the rejection of organ transplants as well as a topical cream for atopic dermatitis. The risks and benefits of the two formulations may differ substantially. Where dosage form or route of administration is relevant to our reporting on a specific drug, it is indicated in that section of the report.
Our analysis focuses on the drug that is identified as the primary suspect drug in the case report. The reports also may name a secondary suspect drug and other drugs being administered concomitantly.
Special Cases of Drug Names
A few drug names and categories present special problems that require a more or less arbitrary solution. A more general category is needed because of a large number of chemically related drugs as well as vague product identification on the original reports. The cases with the greatest effect on results are as follows: All estrogenic drugs are currently combined, including those for contraception and hormone replacement. From time to time we publish detailed breakout showing the brand names that are included in a current quarter results. We also group all interferon products in to the following groups: interferon alfa, interferon beta and interferon gamma. For similar reasons we group together all forms of insulin products.
Special Reporting Drugs
We classify in the special reporting category certain drugs in which the report volume is influenced by special circumstances that appear unrelated to the safety profile of the product. Some drugs are approved only for restricted distribution and have special programs for reporting all adverse events whether or not the drug was suspected of causing the event. Examples include thalidomide, lenalidomide, ambrisentan and deferasirox. Since we know of no central listing of restricted distribution products, drugs are added to this category as we learn of their special status. The category includes other drugs where manufacturer direct contact with patients leads to large numbers of reports where the drug was not necessarily suspected of causing the adverse event. Examples include the peritoneal dialysis solution called Dianeal, the injectable bisphosphonate drug ibandronate (Boniva).
Characterization of Adverse Events
In the FDA adverse event reports released for research, the narrative that described the event that occurred is replaced by one or more standardized medical terms drawn from the Medical Dictionary for Regulatory Affairs (MedDRA).3 The MedDRA terminology was developed by the pharmaceutical industry to standardize international reporting of data from clinical trials and postmarketing adverse events. The dictionary is updated twice a year and QuarterWatch™ updates its version annually. The most granular (or specific) terms in case reports are called Preferred Terms (PT), and number approximately 30,000. One adverse event case might include a single PT term (e.g. nausea) or several terms (e.g nausea, vomiting, and myocardial infarction). QuarterWatch™ analysis also uses two other features of the MedDRA terminology that permit grouping similar or related terms together to provide a more general event description. The terminology provides approximately 1700 High Level Terms (HLTs) that collect together a group of closely related PT terms. For example, 30 different forms of specific intestinal infections are grouped together under the HLT called intestinal infections. HLTs, in turn, are combined into still less granular group of more than 300 High Level Group Terms (HLGTs). At the most general level, System Organ Classes (SOCs) combine the HLGT terms into 26 categories.
The MedDRA terminology provides another tool for grouping together terms that are clinically related (even though not near synonyms) into an umbrella term called a Standardized MedDRA Query (SMQ).4 The SMQs allow grouping terms that are clinical related but functionally different. For example, a liver function laboratory test result (ordinarily classified as an “investigation”) could be grouped with other kinds of terms that might indication a liver disorder. SMQ definitions can be either broad scope or narrow scope; QuarterWatch™ utilizes broad scope SMQs.
Because each case report might contain more than one MedDRA term (on the average more than 3 terms), the problem of arises of what is an “event?” Should it be one case (with several terms) or is each term a separate event. In QuarterWatch™ reports each case can appear in a specific SMQ or HLT only once. However, each case might possibly appear in more than one SMQ. (for example, the term nausea could cause a case to be classified in at least three SMQs.) When reporting on specific Preferred Terms we describe the totals as “mentions” because it means that, for example, 10 reports mentioned the term “nausea.” However, in any list of Preferred Terms it is possible that one case report might contain more than one term on that list.
Product Quality and Medication Error Reports
Special criteria are applied to identify and count reports indicating either a product quality problem or a medication error. The cases are selected using the MedDRA High Level Group Terms (HLGT) specifically designed for these two categories (“Product quality issues”, and “Medication errors”). In addition, these categories include reports that are not serious, which are typically predominant. For product quality complaints, the primary drug identification is by the specified brand name, rather than the generic or chemical name, which could combine products from several different manufacturers.
Report Versions, Counts, and Revisions
Quarterly report totals for individual drugs and categories are subject to variability in the underlying data. The FAERS data may include more than one version of a specific case report. This occurs because manufacturers revise the reports as the companies obtain additional information. We use the most recent revision (called the last best case) that was released, but retain the original initial reporting date. Additional variability in the report totals in QuarterWatch™ comes from several other sources. Each quarterly distribution contains hundreds of reports that were received (and belong in) the previous quarter but were not processed in the FDA system in time to be included in the quarterly totals. Thousands of reports from earlier quarters are revised by the manufacturer. In addition, manufacturers may submit reports for events that occurred in previous quarters—or even previous years—thus inflating one quarter’s totals. These reports from earlier periods usually occur for two reasons. In some cases a company had a reporting problem and submitted reports for an extended period. Also, FDA regulations allow companies to submit Periodic Reports on an annual basis rather than a quarterly basis three years after approval. Finally, QuarterWatch™ criteria are changed from time to time as more is learned about the overall system performance. The consequence of this variability is that any previously reported drug, quarter or event total may change over time. As a result, QuarterWatch™ recalculates the historical and drug trend data every quarter. Therefore investigators seeking to reproduce QuarterWatch™ event totals are likely to obtain results that vary slightly from those reported.
Dispensed Outpatient Prescriptions
To provide a broader perspective on the adverse events reported, we assess the patient exposure to drugs on the basis of U.S. dispensed outpatient prescription data provided by IMS Health Inc. The data we rely on are an estimate of total non-governmental prescriptions dispensed through retail and mail channels. Our agreement with IMS includes the following disclaimer:
“The statements, findings, conclusions, views, and opinions contained and expressed in QuarterWatch™ are based in part on data obtained under license from an IMS Health Inc. information service called the National Prescription Audit™ for 2015 (All Rights Reserved). Such statements, findings, conclusions, views, and opinions are not necessarily those of IMS Health Incorporated or any of its affiliated or subsidiary entities.”
Proportional Reporting Ratio
We use a statistical technique called the proportional reporting ratio5 to measure the strength of the association between reports of an adverse effect and the suspect drug. By comparing the observed number of reports for a adverse effect with the expected number, given the total report volume, it becomes possible to detect a valid signal from a relatively small number of reports, and to discount some cases where reports were more numerous, but more likely a chance effect. Several factors, unrelated to the safety of a drug, might influence the number of reports received. This includes greater or lesser patient exposure, differences in reporting rate, and the play of random chance.
The proportional reporting ratio for an adverse effect calculates the proportion of reports for the adverse effect under study compared with the proportion of reports for the side effect for all other drugs in the preceding four calendar quarters. For example, suppose the adverse effect under study was MedDRA term “Depression.” We would compare the proportion of reports (suspect drug depression cases/suspect drug all cases) with the proportion for all drugs (all depression cases/all reports). For example, if a suspect drug had 10 cases of depression among 100 reports overall, we conclude that the observed proportion was 0.10. For all drugs in 2011 we identified 3450 reported mentions of depression among 179,855 cases overall, or an expected value of 0.019. Therefore the proportional reporting ratio would be 0.1/0.019 or PRR = 5.2. This shows that the suspect drug had 5.2 times the expected value given the reports for all other drugs. The next step is to determine whether this five-fold difference could have occurred by chance. Using the chi-square statistical test (with Yates correction) we calculated that the chi-square value was 40.4 and the probability this association occurred by chance was p < 0.001.
We use the word signal to characterize credible evidence we see of a safety issue. The term signal as used in QuarterWatch™ means evidence of sufficient weight to justify an alert to the public and scientific community, and to warrant additional investigation to assess a causal relationship to the drug and determine its incidence.
The submission of an individual report does not in itself establish that the suspect drug caused the event described—only that an observer suspected a relationship. While the sheer numbers of case reports have scientific weight, because of variation in reporting rates they reveal little about how frequently the events occur in the broader patient population.
- Center for Drug Evaluation and. (n.d.) FDA Adverse Events Reporting System (FAERS) - FDA Adverse Event Reporting System (FAERS): Latest Quarterly Data Files. Accessed 26 March 2014.
- Code of Federal Regulations Title 21 314.80 Postmarketing reporting of adverse drug experiences. (2011) Food and Drug Administration. Accessed 7 April 2014.
- MedDRA MSSO. (2014) Introductory Guide MedDRA Version 17.1. Chantilly, VA: MedDRA Maintenance and Support Services Organization.
- MedDRA MSSO. (2014) Introductory Guide for Standardised MedDRA Queries (SMQs) Version 17.1. Chantilly, VA: MedDRA Maintenance and Support Services Organization.
- Evans SJ, Waller PC, Davis S. (2001) Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf 10: 483–486.