Table 1

Interview questions

Sociotechnical dimension	Interview questions	Rationale
1. Hardware and software Equipment and software required to run the applications	Does your site have any modified software that impacts alert management? Do you generate any reports to monitor the changes made to the software?	The EHR allows facilities to make changes to the software to address local needs, which can affect how alerts are managed
2. Clinical content Data, information, and knowledge entered, displayed, or transmitted in EHRs	The number of mandatory, enabled and disabled alerts? Are there any national or network level mandatory alerts?	The notification management options within CPRS can be used to turn specific notification on or off. Alerts can be enabled, disabled or set as mandatory. Some alerts are mandated centrally. The number of enabled and mandatory alerts can affect alert volume
3. User interface Aspects of the EHR system that users interact with	In a typical workday, how many providers request support? How often do you get calls from providers about missed or lost alerts?	A poorly designed user interface can lead to difficulties in managing alerts, prompting the providers to seek support to manage alerts
4. People Humans involved in the design, development, implementation and use of HIT	How much time is spent on View Alert training? Does the site have specific training on View Alerts?	CPRS uses the ‘View Alert’ notification system to inform clinicians about critical test results. Providers should have necessary training to process view alerts
5. Organisational policies Internal policies and procedures that affect all aspects of HIT management	Does your facility have a policy on test result communication? Do you have an EHR committee for oversight?	Having a test result communication policy is important to ensure that there is no ambiguity regarding acknowledgement and follow-up of alerts
6. State and federal rules External forces that facilitate or place constraints on HIT use	Are you aware of the VHA 2009 directive for communication of test results?	The VHA 2009-019 directives mandates that patients should be notified about all test results within 14 days
7. Workflow and communication Work processes needed to ensure proper functioning of the system	Do you have any mechanisms to prevent alerts from falling through the cracks/alerts being missed? Do you have a case manager that gets notified about certain abnormal results? Are alerts set to go to a team rather than specific provider?	Facilities should have mechanisms in place to make sure that critical alerts are not missed/lost. Back-up procedures to prevent alerts falling through cracks should be implemented
8. Monitoring Measurement of system availability, use, effectiveness, and unintended consequences of system use	What monitoring practices do you have in place for follow-up of critical/abnormal diagnostic test results? Is acknowledgement and follow-up of alerts monitored at your facility?	In order to keep track of critical test result follow-up, good monitoring practices should be in place

CPRS, computerised patient record system; EHR, electronic health record; HIT, health information technology; VHA, Veterans Healthcare Administration; VISN, Veterans Integrated Service Network.