Emergency Preparedness: Be Ready for Unanticipated Electronic Health Record (EHR) Downtime

Problem: In its accreditation manual for hospitals, The Joint Commission (TJC) describes an “emergency” as an incident that often occurs rapidly without notice, impacting the organization’s ability to operate and provide services it considers essential or critical.¹ Unanticipated electronic health record (EHR) downtime is just one of many “emergencies” an organization may face. Unlike its planned counterpart, which is scheduled for system maintenance and upgrades, approved by leadership, and communicated to staff in advance, unplanned downtime is not only unpredictable and more disruptive in nature, but it can last for several hours or even days. While most institutions have established a policy and procedure for scheduled EHR downtimes, many organizations find themselves inadequately prepared for unanticipated EHR downtime events.

Frequency. Unanticipated downtime of information technology (IT) systems is inevitable, despite best efforts to keep them running smoothly. In two studies looking at a 3-year timeframe, one found that 96% of organizations reported at least one unplanned IT system downtime,² while the other study found that 70% of organizations reported at least one unplanned downtime lasting 8 or more hours.³ Furthermore, between 2020 and 2022, the Department of Veterans Affairs (VA) experienced 52 occurrences in which the EHR system had been partly or completely unusable.⁴ A 2015 survey in Finland found that nearly half of 2,864 respondents reported that extended unavailability was the highest perceived risk related to EHR systems.⁵

Direct causes. Unplanned EHR downtime events can be caused by power failures, software failures (partial or full EHR unavailability), system interface failures, computer viruses or malicious software programs, incorrect computer configurations, or wireless connectivity issues. Some events may involve extreme weather conditions and outdated building infrastructure for which the recovery process may be extensive, time-consuming, and associated with longer system recovery times.⁶

Impact. While the full scope and clinical impact of downtime events may not be readily apparent when the event initially occurs, it could result in delayed patient care and heighten the risk of medication-related adverse events. In fact, harmful medication errors have occurred during both scheduled and unplanned downtimes, even in the presence of backup systems and standardized protocols.⁷ A lack of downtime planning and training, resulting in delayed medication ordering, dispensing, and administration have been cited as contributing factors for these harmful medication errors.

Recently, ISMP received reports that highlight the vulnerability that practitioners face during downtime, when there is a loss of technological support to help catch medication errors. Two examples below show how healthcare has become reliant on computer-generated alerts, which are lacking during downtime.

• A pediatric patient experiencing an allergic reaction to a bee sting was prescribed EPINEPHrine intramuscularly (IM) along with dexamethasone orally. Instead of administering dexamethasone orally, the nurse accidentally administered EPINEPHrine orally in addition to an IM dose, ultimately giving the patient two separate doses of EPINEPHrine and no dexamethasone. The error occurred during a scheduled downtime when both the barcode scanning system and the electronic medication administration record (MAR) were unavailable.

• Two patients with documented allergies to haloperidol received haloperidol injections when the drug-allergy warnings in the EHR did not fire. The cause was a missing field in the medication profile. However, a similar medication error could happen during EHR downtime when system-based alerts are unavailable. Neither patient exhibited signs of an allergic reaction, but both received additional treatment (antihistamine and steroid) as a precaution.

Safe Practice Recommendations: For accredited hospitals, TJC standards outline a process for managing EHR downtime in the Emergency Management and Information Management chapters, with a goal of returning organizations to normal operations as soon as possible with no loss of data.^1,8 Without an organized downtime plan, facilities tend to respond to unanticipated EHR downtime in silos, lacking a comprehensive systems approach which leads to poor interdepartmental communication and collaboration. While preparing for this emergency requires detailed and complex planning and attention,⁹ consider the following recommendations in regards to safe medication use:

Assess the risk. Conduct a failure mode and effects analysis (FMEA) to identify risk points when planning for an emergency. In 2017, ECRI published a self-assessment questionnaire, Unplanned Downtime of Health Information Technology Systems, to help organizations recognize vulnerable processes, proactively identify specific threats to patient safety, and anticipate the magnitude of operational disruptions.

Select a response team. Proactively choose an interdisciplinary team to respond to an unanticipated EHR downtime event. Keep in mind, this team will be responsible for making the difficult decision to mobilize the organization into full downtime mode, initiating the steps required to ensure safe operation of the organization, and communicating with organizational leadership. Be sure the team is provided with sufficient authority to direct staff during an emergency. Similar to the Center for Disaster Medicine at Massachusetts General Hospital (MGH), organizations might consider selecting an initial team to assess the downtime event and its operational impact on the organization, and another team that will be called in if the assessment team deems it warranted.⁹ All team members should have a deep understanding of the EHR process and the steps that need to be taken to verify that mobilization to a downtime mode is necessary.

Identify leaders. During off hours when typical leaders are not on duty, organizational policies should include a designated “on-call” leader for the organization as well as departmental “on-call” leaders who can be contacted in the event of EHR downtime. These leaders should be familiar with the organization’s emergency operations plan in case the full plan needs to be activated. For this role, communicating with and gathering feedback from frontline staff, and diverting or bringing in additional resources, is essential.

Establish a communication triage procedure. Establish a comprehensive communication triage procedure and checklist for notifying all impacted areas regarding the EHR downtime event, how long it is expected to last, and interim steps to be taken until the issue has been resolved. Select key locations for an organizational incident command center from which the assessment and/or response team(s) will initiate and maintain the communication triage procedure and through which leadership updates will be provided. Also plan a robust emergency staff recall system, such as text messages with required confirmation from staff to document availability.

Develop an emergency readiness binder. To address medication safety concerns, compile an organized emergency readiness binder, available electronically and in a physical form (hard copy). The binder should prominently display the version date. Keep the binder in a strategic and well-known location, and review it for accuracy at least annually. In the binder, address the following medication-safety questions or scenarios for when your EHR goes down:

• Affected technology. What other systems could be impacted? Does this impact your telephones, intravenous (IV) workflow system, pharmacy robotics, automated compounding devices, automated dispensing cabinets (ADCs), smart infusion pumps, or barcode scanning technology? Is there a backup system for patient diagnostic results that need to be reviewed? How can each member of the healthcare team review the patient’s test results? As part of this initial assessment, organizations should decide whether a backup server should be in another geographic location, especially if the organization is located in a natural disaster area. Also, if staff are logged into the EHR when the system goes down, there may be a period in which they continue to have EHR access—staff should be instructed not to log out and to communicate their current EHR access to the assessment and/or response team. 

• Needed documents. What documents associated with medication use, including drug information references and policies and procedures (with the version date documented), will be needed in the emergency readiness binder? Examples include patient care flowsheets; medication administration record (MAR) forms; standard, disease-specific, or care area-specific medication order forms; test and procedure forms; nursing-pharmacy communication forms; compounding master formulation records and batch sheets; titration sheets/protocols; drug location maps of medication carousels or robots (which may change frequently); and other critical documents, which can also be secured in cloud storage and backed up regularly into a downtime file repository on specially identified computers on each unit. Practitioners will also need access to drug information databases, which may be available via smart phone apps, and key medication-related standard operating procedures. While some organizations have moved away from paper documents, if there is a power failure, maintaining critical documents as physical copies is essential.

• Transmission of orders. How will written orders be sent to the pharmacy to ensure they are acted upon and do not contribute to delays in therapy? Will they be faxed or hand-delivered? Are there situations in which verbal orders will be allowed? Where will standard, disease-specific, or area-specific order forms be stored during EHR downtime? The expectation should be to minimize verbal and telephone orders and to enforce the use of written order forms during EHR downtime.

• Order verification. Since clinical decision support is unavailable during EHR downtime, prescribers, pharmacists, and nurses should seek answers to the following questions for manual screening: Does the patient have an allergy to the medication? Is the drug appropriate based on the patient’s diagnosis? Are the dose, route, and frequency appropriate based on the patient’s age, weight, and renal and liver function? Are there other pertinent lab values that should be reviewed first? Are there drug-drug and/or disease-drug interactions? Is the prescribed medication a therapeutic duplicate of another medication prescribed for the patient? Include these questions in the binder and require manual screening during EHR downtime.

• Dosing. If dosing calculators and nomograms are normally part of the EHR, consider maintaining these tools on a standalone computer, and access this computer to obtain dose calculations rather than allowing staff to manually calculate doses, which they may not have done for some time or ever.

• Dispensing. How will pharmacy staff communicate with the cleanroom while maintaining a sterile environment? Convene a team huddle outside of the cleanroom, initially and then as needed, to clarify processes and troubleshoot issues. How will medications be labeled, ensuring all required components? Purchase label printers connected to software outside of your EHR, and/or prebuild label templates for frequently ordered medications and complex high-risk medication orders to avoid the need for pharmacy staff to manually enter the label information via free text. How will patient-specific oral medications, including pediatric oral solutions, be dispensed? How will unit and ADC stock be replenished, and how will ADC medications be stored on units if normally kept in an ADC that is also down? If you use carousels, think about how you would efficiently fill patient-specific medications manually if your carousels are no longer linking to the EHR, not able to be navigated utilizing the computer systems, or not able to spin on their own. How will pharmacy keep track of the number of doses that have been dispensed? Consider making a photocopy of the label and placing it into a binder, documenting the number of doses dispensed to account for this, and to gather patient charges.

• Administration. If the EHR loses connectivity to the ADC, will ADCs automatically be placed on critical override status for immediate access by the units, or is this a manual process that must be activated? For organizations that lose interoperability between the EHR and smart infusion pumps, have nurses been educated about manually programming the pump? Since bedside barcode scanning technology will be disabled, are nurses prepared to revert to manual confirmation of the correct patient and medication? Consider updating nursing education as needed and implementing certain safeguards (e.g., checklists; independent double checks of transcription, the dose/concentration/rate of infusion, and/or the label; partnering with the patient), particularly for at-risk patient populations and high-alert medications. 

• MARs. If the EHR is not accessible in real-time, what is the process to retrieve the most recent MAR and pertinent patient information? Will the MAR be printed, who is in charge of printing it, and where will the printed copy be stored? Some organizations have access to a downtime viewer that continuously backs up electronic MARs and can be accessed at designated computers. Review the information that prints on the MAR downtime report to evaluate if changes are necessary. 

• Medication reconciliation. What will the medication reconciliation process look like for new admissions? To make it easier for staff to collect a medication list and conduct medication reconciliation, consider creating an admission form with prompts to collect pertinent patient information (e.g., age, sex, diagnoses, allergies and sensitivities, metric height and weight, pregnancy and lactation information, laboratory results). Include scripted questions to ensure a complete list of current medications (including supplements, herbals, and over-the-counter products) is initially collected. Keep this admission form and scripted questions in the emergency readiness binder.

Educate staff and test the response. What education is needed regarding the EHR downtime plan and emergency readiness binder for departmental leaders, the response team, IT staff, and EHR users? When should educational modules and refresher classes be provided, and how will annual competencies be developed to reflect downtime plans and procedures? What type of simulations and practice exercises would be helpful in teaching leaders and individual departments to respond? Planned EHR downtime should be used as a learning experience to uncover issues that must be addressed,¹⁰ assess knowledge, and ensure the adequacy of policies and procedures. On July 1, 2022, TJC implemented a requirement to conduct two exercises per year to test the emergency operations plan,⁸ and we recommend including EHR downtime as part of these exercises. Departmental unit inspection checklists should also include a review of the emergency readiness binder material, confirming that required and updated documents are available.  

Initiate the process for events. In the event of unanticipated EHR downtime, trigger the designated team to respond immediately to determine the extent and anticipated timeframe for correction of the issue, determine potential solutions, and ultimately to decide if EHR downtime procedures should be implemented. The initial and active Downtime Assessment and Response Teams (iDART, DART) used at MGH have specific duties described below⁹:

• An initial team (iDART) is notified of the EHR downtime event in real-time and assesses the operational impact of the event. The team then decides what type of response, if any, is warranted.

• If iDART determines that the downtime event will have an impact on hospital operations, the response team, DART, is activated to manage the event.

• DART directs staff to manage the event via the Incident Management Team or (Hospital) Incident Command System and activates the hospital’s emergency operations plan if significant downtime is expected.

Recovery. When the EHR is back up, how will downtime orders be documented retrospectively in the EHR? What is the threshold for which orders will be transcribed and backdated into the system compared to those that will be scanned to become part of the permanent medical record? While there should be organization-specific defined timeframes, unless the system has been down for multiple days, we encourage all orders to be directly entered into the EHR immediately to ensure communication of drug information and prevent medication errors. Special attention should be given to documenting the orders that have not yet been completed. Also, medication errors identified during unanticipated or anticipated EHR downtime should be analyzed to determine if practice changes are needed. Soon after the recovery, convene a multidisciplinary group of those impacted by the downtime to determine what system changes should be developed and implemented in anticipation of the next unscheduled downtime.

Conclusion: As with other unanticipated or catastrophic events, unanticipated EHR downtimes often lead to chaos for both leaders and frontline EHR users, especially if staff are not prepared and do not know what to do. These situations require leadership support as well as organizational and departmental policies and procedures. Resources supplied in the emergency readiness binder should be carefully thought out and vetted through appropriate committees, and staff need to know how to access this information. The unanticipated EHR downtime policies and procedures (and emergency binder) should be reviewed at least annually and updated as needed based on changes in protocols, available literature, and feedback from staff during practice drills. We encourage organizations to let us know ([email protected]) about their experiences with unanticipated EHR downtime so that we can share the lessons learned with other organizations.

References

1. The Joint Commission. E-dition (web-based access to Comprehensive Accreditation Manuals). 2022.
2. Sittig DF, Gonzalez D, Singh H. Contingency planning for electronic health record-based care continuity: a survey of recommended practices. Int J Med Inform. 2014;83(11):797-804.
3. Ponemon Institute. Sixth annual benchmark study on privacy & security of healthcare data. May 2016.
4. Diaz N. VA Cerner EHR system has been ‘unusable’ 52 times since 2020. Becker’s Health IT. May 9, 2022.
5. Palojoki S, Pajunen T, Saranto K, Lehtonen L. Electronic health record-related safety concerns: a cross-sectional survey of electronic health record users. JMIR Med Inform. 2016;4(2):e13.
6. Chen J, Wang Y, Magrabi F. Downtime in digital hospitals: an analysis of patterns and causes over 33 months. Stud Health Technol Inform. 2017;239:14-20.
7. Hanuscak TL, Szeinbach SL, Seoane-Vazquez E, Reichert BJ, McCluskey CF. Evaluation of causes and frequency of medication errors during information technology downtime. Am J Health Syst Pharm. 2009;66(12):1119-24.
8. The Joint Commission. New and revised standards in Emergency Management. R3 Report. Requirement, Rationale, Reference. 2021;34:1-10.
9. Massachusetts General Hospital Center for Disaster Medicine. Hospital preparedness for unplanned information technology downtime events: a toolkit for planning and response. July 2018. 
10. Vaughn S. Planning for system downtimes. Comput Inform Nurs. 2011;29(4):201-3.

^{Suggested citation:}

^{Institute for Safe Medication Practices (ISMP). Emergency preparedness: be ready for unanticipated EHR downtime. ISMP Medication Safety Alert! Acute Care. 2022;27(17):1-5.}