Article Text
PDF
Abstract
Introduction In primary care, almost 75% of outpatient visits by family doctors and general practitioners involve continuation or initiation of drug therapy. Due to the enormous amount of drugs used by outpatients in unmonitored situations, the potential risk of adverse events due to an error in the use or prescription of drugs is much higher than in a hospital setting. Artificial intelligence (AI) application can help healthcare professionals to take charge of patient safety by improving error detection, patient stratification and drug management. The aim is to investigate the impact of AI algorithms on drug management in primary care settings and to compare AI or algorithms with standard clinical practice to define the medication fields where a technological support could lead to better results.
Methods and analysis A systematic review and meta-analysis of literature will be conducted querying PubMed, Cochrane and ISI Web of Science from the inception to December 2021. The primary outcome will be the reduction of medication errors obtained by AI application. The search strategy and the study selection will be conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses and the population, intervention, comparator and outcome framework. Quality of included studies will be appraised adopting the quality assessment tool for observational cohort and cross-sectional studies for non-randomised controlled trials as well as the quality assessment of controlled intervention studies of National Institute of Health for randomised controlled trials.
Ethics and dissemination Formal ethical approval is not required since no human beings are involved. The results will be disseminated widely through peer-reviewed publications.
- PUBLIC HEALTH
- Risk management
- PRIMARY CARE
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Robust and reproducible process for systematic review of the literature according to the guidelines of Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Meticulous studies quality assessment will be carried out.
There could be an eventual lack of homogeneity of patient safety indicators.
There might be a low and inconsistent quality of included studies.
Introduction
Patient safety is a global public health issue. Adverse drug events and medication errors are frequent preventable causes of increased morbidity, mortality, hospitalisation rates and healthcare costs.1 According to Williams, medical errors can be subgrouped into three classes: prescribing, dispensing and administration errors.2 The preventability of these events is a critical factor that must be carefully interpreted. Indeed, the evaluation of causal inference is often critical because adverse events can be associated both to adverse drug reactions (usually unpreventable) and to medication errors (preventable since related to human decisions).3 According to the Institute of Medicine, 1 of 131 outpatient and 1 of 854 inpatient deaths are caused by medication errors.4 However, there is little evidence on the real incidence of errors, especially in primary care setting.5 Primary care is a complex system composed by healthcare professionals—working within socio-sanitary structures—that provide first medical care for acute diseases and guarantee continuity of assistance in chronic pathologies. Three-fourths of the visits of family doctors concern the indication or the follow-up of a pharmacological treatment.6 Outpatients are significantly more exposed to a risk of drug misuse (and thus of adverse events) than inpatients because of the lack of a strict medical monitoring.7 Since primary care is a heterogeneous and complex setting and—as said—drug-related errors are extremely frequent, artificial intelligence (and, in particular, e-health) could have a significant impact on the safety and the quality of care in this field. In particular, the clinical decision-making can be supported and empowered by algorithms.8 Artificial intelligence (AI) can be defined as ‘the science and engineering of making intelligent machines, especially intelligent computer programmes’.9 In medicine, it can be used for the improvement of patients’ diagnosis, management and treatment. In particular, machine learning (ML) is the main branch of AI, being involved in the development of algorithms based on big data. The main implications of ML models concern various fields of medicine: that is, prediction in clinical and community care settings of chronic diseases, decision-making behaviours, clinical decision support and enhancement of efficiency of medical imaging.10 In particular, electronic health records and electronic support systems based on algorithms could enhance the compliance with standards, avoid preventable errors and tailor the treatment on the basis of the specific characteristics and needs of the patients.11 12 The aim of the study is to investigate the impact of AI algorithms on drug management in primary care settings. Furthermore, we aim to compare AI or algorithms with standard clinical practice to define the medication fields, where a technological support could lead to better results.
Methods and analysis
The synopsis for this systematic review is prospectively submitted in the International Prospective Register of Systematic Reviews.
Important amendments and updates made to the protocol will be documented and published alongside the results of the systematic review.
Search strategy
A comprehensive search strategy will be created and implemented according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis Protocols checklist.13 The population, intervention, comparator and outcome (PICO) framework14 was adopted to formulate the following research question: ‘Do Artificial Intelligence and Algorithms in primary care have the potentiality to disrupt patient care with a safer and faster medication management?’. MEDLINE (via PubMed), Cochrane and ISI Web of Science databases will be queried to retrieve relevant peer-reviewed articles. Initially, controlled descriptors and the relative keywords were identified and verified in each scientific database. Afterwards, a Boolean search string, combining Medical Subject Headings terms and free-text words such as ‘primary care’, ‘ambulatory care’, ‘outpatient care’, ‘general practitioner’, ‘general paediatrics’, ‘artificial intelligence’, ‘algorithms’, ‘machine learning’, ‘deep learning’, ‘neural networks’, ‘medication error’, ‘adverse event’, ‘prescribing error’, ‘dispensing error’, ‘administration error’, ‘monitoring error’, ‘medication errors reporting’ and ‘medication reconciliation’ will be used. In addition, the reference lists of all relevant articles and the references for additional data sources missed during the database search will be scanned (ie, snowball search) and their full texts will be retrieved. The accuracy of the search strategy will be determined by preliminary checking for relevant studies to retrieve all the appropriate terms and synonyms, then by asking the specialists’ opinions15 to increase the robustness of the selected terms and synonyms, and, finally, by using the Peer Review of Electronic Search Strategies checklist.16 The full search strategy is available in the online supplemental file S1.
Supplemental material
Study selection criteria
All the articles that will meet the inclusion criteria will be included in the systematic review. Table 1 provides a brief summary of the main elements considered in the PICO model.
Inclusion and exclusion criteria
Additionally, the inclusion will be restricted to original primary analyses written in English describing randomised controlled trial (RCT), clinical trials or controlled trials. Thus, systematic reviews and meta-analyses will be excluded. The search strategy will be also restricted by availability of full texts published in peer-reviewed journals. Articles not focusing on digital technologies-based AI interventions will be excluded.
The primary outcome measures will be the reduction of preventable medication errors that resulted in a decrease in hospital admissions, emergency department visits and mortality through the application of AI or algorithms to primary care settings.
The secondary outcome will be the identification of the medication fields, where technological support could lead to better results.
Screening and data extraction
After the removal of duplicate articles, and according to the inclusion and exclusion criteria, four independent researchers (FC, GA, MTR and MZ) will conduct the initial screening by evaluating the titles and abstracts. Then, the same researchers will screen the full text of each study to determine the potential eligibility. In both of the two screening phases, any disagreements or ambiguous situations will be resolved by a fifth author by discussing the inclusion and exclusion criteria of the article.
Data extraction will be completed by three independent investigators (MCN, SG and MS). A data extraction spreadsheet will be designed, including the following: (1) study characteristics (ie, first author, publication year, country of the study, journal title and article title); (2) setting characteristics (ie, home setting, ambulatory and nursery home); (3) methodological characteristics (ie, study type, duration of intervention, sample size, target population, type of medication error, type of intervention and comparator); and (4) the main findings (ie, outcomes, quadruple aim and severity of avoided reaction).
Quality assessment
Methodological quality of the RCTs and non-randomised controlled trials (NRCTs) will be assessed using the quality assessment of controlled intervention studies of National Institute of Health. This tool analyses several aspects of the included studies: population and participation rate, inclusion criteria, sample size justification, association between exposure and outcome, outcome description, drop-out rate, exposure measures and assessment, and confounding variables. The tool assesses 14 parameters for evaluating the internal validity of a study. For each item, the investigator could select ‘yes,’ ‘no,’ or ‘cannot determine/not reported/not applicable’.
The scale assesses the following study-level aspects: randomisation, allocation concealment, blinding, completeness of outcome data and selective outcome reporting, drop-out rate, adherence to the intervention and dimension of sample size.
A potential risk of bias was considered if the item was rated as ‘no’ or ‘cannot determine/not reported/not applicable’ were selected for the items by the reviewer. If the ‘yes’ answers were ≥75% of the total, an article was considered of ‘good’ quality; if they were <75% but ≥50%, an article was scored as ‘fair’; if they were <50%, the article was scored as ‘poor’. So, a score of 10 or greater was indicative of good methodological quality, 9–7 was fair and studies scoring below 7 were deemed to be of poor quality.
Three reviewers (GA, MCN and GA) will assess independently the quality of included studies and disagreements will be resolved by a fourth reviewer (GS).
Descriptive analysis and meta-analysis
A narrative synthesis, including tables and figures, will be carried out for all the included manuscripts. If applicable, the pooled mean difference and 95% CI will be calculated to abridge continuous data,17 while a proportion of meta-analysis will be carried out for proportion outcomes. Separate pooled analyses will be performed for each group of studies (ie, RCTs and NRCTs) as well as for each included outcome. Furthermore, in case of small numbers of studies will be found, the estimates for per cent reduction in medication errors will be pooled. To deal with potential heterogeneity, a random-effects meta-analysis will be conducted.18
The I2 statistic, which quantifies the degree of variability among studies due to heterogeneity rather than sample error,19 as well as forest plot will be used to assess the heterogeneity.
In addition, there could be many decision nodes, such as the search of studies, eligibility criteria, type of data to be extracted and the type of analyses, within the systematic review process, that may require sensitivity analyses. A leave-one-out sensitivity analysis will be performed by iteratively removing one study at a time to point out if one study may influence the overall estimate of the rest of the studies. If results will be consistent across the different analyses, the findings will be treated as robust while, on the contrary, they need to be assessed with caution.
Publication bias for each outcome will be assessed, if at least 10 studies will be included in the meta-analysis, through funnel plots, and the asymmetry of funnel plots will be tested using Egger’s test.
The main limitations related to the meta-analysis are the biases affecting primary studies, publication, reporting and selection bias, and the heterogeneity.20 Nonetheless, quality assessment and sensitivity analyses will be conducted, by two independent authors, to overcome these caveats.
All statistical analyses will be conducted using statistical software STATA21 V.16 and two-sided p values <0.05 will be considered statistically significant.
Patient and public involvement
No patient involved.
Discussion
We believe that the main implication of this systematic review could be the prioritisation of the medical areas in which a better result, intended as medication error reduction, has been found, creating guidelines that could make the process more efficient. In addition, given the impending PNRR allocation of fundings for technological infrastructures, consolidating the already existing evidences addressing the efficacy of AI could help political decision-makers in allocating the resources. Finally, this systematic review might result an important tool for giving answers to some of the still existing questions relatively to safety and usability of AI machines in the health sector.
Ethics and dissemination
Formal ethical approval is not required since the systematic review and meta-analysis will not foresee the involvement of human beings. The results will be disseminated widely through peer-reviewed publications.
Ethics statements
Patient consent for publication
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
AO and GA are joint first authors.
SG and GD are joint senior authors.
Twitter @Antonio Oliva
AO and GA contributed equally.
Contributors The study concept was developed by AO, GD, GDM and MC. The manuscript of the protocol was drafted by GA, MCN, SG, MZ, GS, FC, GA, MS and MTR, and critically revised by AO, GD, GDM and MC. AO, GD, GDM and MC developed and provided feedback for all sections of the review protocol and approved the final manuscript. The search strategy was developed by GA, MCN, SG, MZ, GS, FC, GA, MS and MTR. Study selection will be performed by FC, GA, MTR and MZ. Data extraction and quality assessment will be performed by GA, MCN and GA, with GS as a fourth party in case of disagreements. All authors have approved the final version of the manuscript.
Funding This study is supported by Fondi di Ateneo, Linea D3.2-Project 'Funzioni pubbliche, controllo privato. Profili interdisciplinari sulla governance senza governo della società algoritmica', Università Cattolica del Sacro Cuore (grant number: R1024500180).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.