Article Text
PDF
Abstract
Objective To evaluate the clinical efficacy and effectiveness of using 3D printing to develop medical devices across all medical fields.
Design Systematic review compliant with Preferred Reporting Items for Systematic Reviews and Meta-Analyses.
Data sources PubMed, Web of Science, OVID, IEEE Xplore and Google Scholar.
Methods A double-blinded review method was used to select all abstracts up to January 2017 that reported on clinical trials of a three-dimensional (3D)-printed medical device. The studies were ranked according to their level of evidence, divided into medical fields based on the International Classification of Diseases chapter divisions and categorised into whether they were used for preoperative planning, aiding surgery or therapy. The Downs and Black Quality Index critical appraisal tool was used to assess the quality of reporting, external validity, risk of bias, risk of confounding and power of each study.
Results Of the 3084 abstracts screened, 350 studies met the inclusion criteria. Oral and maxillofacial surgery contained 58.3% of studies, and 23.7% covered the musculoskeletal system. Only 21 studies were randomised controlled trials (RCTs), and all fitted within these two fields. The majority of RCTs were 3D-printed anatomical models for preoperative planning and guides for aiding surgery. The main benefits of these devices were decreased surgical operation times and increased surgical accuracy.
Conclusions All medical fields that assessed 3D-printed devices concluded that they were clinically effective. The fields that most rigorously assessed 3D-printed devices were oral and maxillofacial surgery and the musculoskeletal system, both of which concluded that the 3D-printed devices outperformed their conventional comparators. However, the efficacy and effectiveness of 3D-printed devices remain undetermined for the majority of medical fields. 3D-printed devices can play an important role in healthcare, but more rigorous and long-term assessments are needed to determine if 3D-printed devices are clinically relevant before they become part of standard clinical practice.
- printing, three-dimensional
- health care evaluation mechanisms
- medical devices
- personalised healthcare
- fabrication
- additive manufacturing
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY 4.0) license, which permits others to distribute, remix, adapt and build upon this work, for commercial use, provided the original work is properly cited. See: http://creativecommons.org/licenses/by/4.0/
Statistics from Altmetric.com
- printing, three-dimensional
- health care evaluation mechanisms
- medical devices
- personalised healthcare
- fabrication
- additive manufacturing
Strengths and limitations of this study
This is the first rigorous systematic literature review of three-dimensional printing for clinical uses.
Validated quality assessment and clinical level of evidence tools are used to assess the progress made in different medical fields.
The study is limited to a critical appraisal of individual studies, rather than a meta-analysis, because of the breadth of uses (from anatomical models and surgical guides to therapeutic devices) and the lack of comparable hypotheses.
Due to the speed of innovation in the field, the review will need to be updated frequently.
Introduction
Three-dimensional (3D) printing is likely to play a pivotal role in transforming healthcare and clinical practice because it provides the opportunity to create customised devices designed for the complexity and individual variances of the patient populations.1
Additive manufacturing, more commonly referred to as 3D printing, is an industrial production technique that enables a 3D digital model to be converted into a physical model by printing it layer by layer. For decades, 3D printing has been used for rapid prototyping, but the recent advances in the available materials, speed, resolution, accuracy, reliability, cost and repeatability of 3D printing technologies have broadened the possibilities for clinical uses.2 3 Many medical fields are already using 3D printing to manufacture custom surgical tools, guides, dose delivery devices, implants, external prosthetics or orthotics and devices for preoperative planning or education.2 4 5 Tailoring devices and procedures to the patient are expected to reduce the time required for surgery, treatment or recovery, while increasing the accuracy and success of the outcome.1 The worldwide 3D printing industry’s revenue from products and services is over US$4 billion and fast growing, with 13.1% of the industry attributed to the medical sector.6
In most medical fields, 3D printing applications are still in the research and development stage or have only just entered clinical practice within the last decade, and hence, there has been a lack of research into the clinical efficacy, effectiveness and long-term follow-up in comparison to traditional technologies.7 Efficacy refers to the performance of the device under ideal and controlled conditions, and effectiveness refers to its performance under typical clinical conditions.8 The Food and Drug Administration states that 3D-printed medical devices are required to meet the same regulations as their non–3D-printed counterparts. Because 3D-printed devices can have different safety and efficacy issues than the equivalent devices, additional testing may be required to demonstrate safety, efficacy and effectiveness.9 Hospitals and clinics will need to adopt new medical product procedures when they want to introduce 3D printing for healthcare, and more evidence of device efficacy and effectiveness will help make an informed discussion before prescribing 3D-printed devices for patients. Providing a critical appraisal of the efficacy and effectiveness of 3D-printed medical devices gives healthcare professionals a resource to assess the validity of the devices and provides researchers with an overview of areas that require further research and validation.
Previous systematic reviews summarise 3D-printed medical devices being used in specific medical fields, such as plastic and reconstructive surgery10 and preoperative planning for liver resections,11 and increasingly available 3D printing processes used in dentistry and mandibular reconstructions.12–14 Systematic reviews on the advantages of 3D-printed devices over conventional methods in surgery have found improved clinical outcomes and reductions in operating times and manufacturing costs.5 15 However, the reviews do not critically appraise the studies and are therefore subject to bias. The only systematic review that did assess bias for a very specific field found no difference in patient outcomes when 3D-printed instrumentation for total knee replacements was compared with conventional instrumentation.16 This highlights the need for a systematic review that incorporates a critical appraisal of the studies that are included.
This review aims to assess the clinical efficacy and effectiveness of 3D-printed devices through performing a systematic literature search, categorising reports of 3D-printed device usage by medical field and purpose and assessing the reports’ scientific quality. The key findings of studies that compare 3D-printed medical devices with their non–3D-printed counterparts are presented, and the gaps in research were highlighted.
Methods
Search strategy
The applied protocol (not previously registered) used systematic methods to search for relevant studies, screen them for eligibility and assess their quality. The review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines.17 A literature search was performed using PubMed, Web of Science, OVID, IEEE Xplore and Google Scholar. A combination of 45 relevant keywords were used to collect all studies that included a 3D-printed device and a clinical trial. Due to the word and character limit search restrictions in IEEE Xplore and Google Scholar, a narrower search of 15 terms was performed for these databases. The search includes all publications up to January 2017. The complete search strategy is provided in the online supplementary material. Mendeley was used as a reference manager.
Study selection
After removing all duplicates from the databases, the title and abstract of each publication was reviewed using a double-blinded method (undertaken by LED and JHMB) to determine eligibility for inclusion.
The inclusion criteria were:
Relevance: papers were required to report first-hand on the results of a clinical study that assessed the efficacy and effectiveness of a 3D-printed device.
Language: only papers written or translated into English were included in the review.
Peer review: records that had no peer review or where the level of peer review could not be traced were excluded from the review.
Where the results of the reviewers (LED and JHMB) conflicted, the reviewers discussed their reasoning for inclusion/exclusion, and where required for clarification of the paper’s relevance, the paper’s introduction and methods were assessed. Consensus on eligibility was achieved for all papers. Included articles underwent a full-text review, and references cited in these studies were also examined for relevance using the same inclusion criteria.
Quality assessment
The studies were then rated according to their level of evidence, based on the Oxford Centre for Evidence-based Medicine Levels of Evidence,18 as summarised in table 1. Systematic reviews and opinion papers were excluded because they cannot provide the original data in a suitable format to be assessed using the quality assessment method used in this review. Where studies used 3D-printed devices but the hypothesis, aim or objective was focused on an outcome other than the effect of the 3D-printed device, the paper has been graded as the level of evidence shown for the device. All levels of evidence were included in the review, but only those that used a control group (levels 1–3) were critically analysed.
Centre for Evidence-based Medicine Levels of Evidence
Studies were then placed in medical fields based on the International Classification of Diseases chapter divisions19 and categorised by the purpose of the device. The three categories were devices used for preoperative planning, devices to aid surgery and therapeutic devices.
To analyse the quality of the studies under review, critical appraisal tools were assessed to find one that met the following criteria:
Suitable for assessing the quality of both randomised and non-randomised studies,
Well regarded and commonly used for quality assessments in systematic reviews.
Demonstrates internal consistency, test–retest reliability and criterion validity.
Assesses the risk of bias, the participant selection methods, the study protocol and the validity, reliability and responsiveness of the study’s results.
Simple and intuitive to interpret.
Does not assign weights to the items in the scale where there is a lack of empirical evidence to support the assignment.20
The Downs and Black Quality Index21 meets all but the final criterion on this list. To overcome this limitation, responses have been left in their raw form (yes/no/unable to determine/not applicable, in accordance with the definitions provided by Downs and Black21) and presented in a table to enable the reader to visualise the category trends, as recommended by the Cochrane Handbook for Systematic Reviews of Interventions.20
The study details and data relating to the quality of study design and reporting were manually extracted from each paper by one reviewer (LED). The study details included medical field, study design, device category, device purpose, number of participants, age and gender of participants and aims and outcomes of the study. The data relating to study design and reporting were extracted according to the criteria used by the Downs and Black Quality Index, divided into reporting, external validity, bias, confounding and power. Where there were uncertainties as to how the study fitted with the Downs and Black Quality Index criteria, the other two authors were consulted, and in all cases, an agreement was reached.
Results
The search yielded 4505 records, and after removing duplicates, 3084 abstracts were screened. A total of 350 studies met the inclusion criteria and were included in the review (figure 1).
Flow chart of the selection and sorting method.
Due to the large number of included studies, this paper gives an overview of the level of evidence found for 3D-printed medical devices in each medical field and then focuses on the outcomes of the level 1 studies. Information for all studies included in the review and a critical appraisal of all comparative studies is provided in online supplementary material.
A wide range of 3D-printed devices have been developed and clinically trialled. The most common device types were anatomical models for preplanning surgeries (40.9%), followed by guides to aid in surgery (37.1%). The studies covered nine medical fields, with the majority (58.3%) of studies falling into the oral and maxillofacial surgery field, which included dentistry and orthopaedic surgery of the jaw, face and skull, and those covering the musculoskeletal system (23.7%) making up the second group. Some studies spanned multiple medical fields, such as neoplasms that occur in the circulatory system. In these cases, the authors selected the most prominent field of the paper. Only 14.0% of included studies used a control group (levels 1–3 in figure 2), whereas 41.4% were level 5 studies. The average number of participants per controlled trial was 41 (36 for randomised controlled trials (RCTs)), compared with 17 for all studies.
Number of studies per level of evidence given by medical field. Levels of evidence are colour coded and fields are visually separated by alternating between white and grey columns.
All 21 level 1 studies fell within the oral and maxillofacial surgery or musculoskeletal system categories, with one cross-field study fitting in both oral and maxillofacial surgery and neoplasms (table 2 summarises all RCTs). All level 1 studies, with the exception of Stephens et al,22 used objective measures as their indicators of device efficacy and effectiveness. Of these, the seven models used for preoperative planning were all modelling bone or fracture fixation plates, though the locations spanned skull, face, mandible, arm, spine, hip and ankle. All aimed to assess the effectiveness of using the models for surgical planning in treating specific conditions. Six out of seven studies used operating time, four used accuracy of the surgery and three used blood loss as key indicators of device effectiveness. Aesthetics, recovery time and postsurgery function were also used as indicators of device effectiveness. All level 1 studies of 3D-printed models for preoperative planning that used time, accuracy or blood loss as indicators of device effectiveness reported that the test group had better outcomes than the control group, with statistical significance at P<0.05 found for four studies that assessed operating time, three studies that assessed accuracy and two studies that assessed blood loss. The two studies that assessed recovery time found no difference between groups, one of the two studies that assessed user function found statistically significant differences in the test group and the other found no difference between groups.
Summary of randomised controlled trials (type: p=preoperative planning, S=surgical tool or guide, T=therapeutic device)
Of the 11 RCTs that assessed 3D-printed guides or templates that supported navigation and surgery (of the mandible, teeth, shoulder, spine, hip or knee), 9 studies used accuracy of positioning as an indicator of device effectiveness, all of which found an improvement when using the guide or template and 8 of which showed statistically significant results at P<0.05. All four studies that used time as an indicator of device effectiveness showed decreased operating times when using the guide or template (all with P<0.05), and five of the six studies that used changes at the operation site (blood loss, inflammatory response, tactile sensation or aesthetics) found statistically significant improvements at P<0.05.
Only three therapeutic devices were tested using RCTs. These were zirconia crowns, which were found to have a better marginal gap and internal fit than the conventional crowns tested (P<0.05); cranial implants, which showed less vertical ridge resorption over time (P<0.05); and osteosynthesis plates for intercondylar humeral fractures, which showed a reduction in operating time (P<0.05) but no difference in elbow function during recovery.
While most level 1 studies found statistically significant improvements at P<0.05 when using the 3D-printed device, none performed an analysis on the clinical relevance of these improvements.
Critical appraisal of the RCTs from each medical field is shown in table 3. The table includes the abridged quality index questions. Studies with higher levels of evidence typically rated better across all areas assessed using the quality index. However, many of the RCTs did not demonstrate external validity and did not blind participants or those measuring the outcomes or assess confounding factors between groups. Only 57% of the RCTs included follow-up.
Critical appraisal of studies with the highest level of evidence from each medical field
Discussion
The results show that a wide range of 3D-printed devices have been clinically trialled, but few papers have rigorously assessed the efficacy or effectiveness of clinical 3D-printed devices. The 3D-printed devices tested were mainly designed for planning surgeries, particularly using anatomical models or surgical guides to aid cutting or navigating, with little attention dedicated to 3D-printed interventions. This is likely due to the lower complexity of the anatomical models and surgical guides and the smaller risk to the patient than therapeutic devices, resulting in less rigorous regulation and safety testing requirements. Approximately twice as many studies were in oral and maxillofacial surgery than in the musculoskeletal system, the two largest fields, but the musculoskeletal system, despite its more recent uptake of 3D printing, had almost double the number of level 1 studies. This is perhaps because clinical trials in musculoskeletal surgery typically aim to demonstrate quantitative clinical outcomes, driving more rapid growth in the level of evidence of studies than oral and maxillofacial surgery where aesthetics is a priority and individual outcomes are harder to quantify.
The RCT listed under neoplasms was an anatomical model used for planning mandibular surgery, but with a focus on tumour removal, fitting into both categories. Therefore, no RCTs were found outside the oral and maxillofacial surgery or musculoskeletal system fields. The primary indicators of success used to determine the efficacy and effectiveness of 3D-printed devices across all medical fields were operating time, accuracy of surgery or positioning, fit, changes at the operation site (particularly blood loss, inflammation and aesthetics), recovery time and functional outcomes. From the studies that compared against a control, the 3D-printed devices used for preoperative planning and aiding surgery consistently found decreases in operating time and increases in surgical accuracy, the two most commonly reported indicators of effectiveness. Operating time is an obvious choice as an indicator for device success because it is easy to measure and quantify and corresponds to decreased blood loss and faster recovery. The main aims of anatomical models and surgical guides are to provide better information to the surgeon on the surgical site and to guide the surgeon’s hand. Therefore, measuring the accuracy of the surgeries indicates whether these aims have been met. To make more decisive conclusions regarding the performance of 3D-printed medical devices in oral and maxillofacial surgery and musculoskeletal surgery, it is important to assess the long-term effects. Few papers assessed the long-term differences of using the 3D-printed devices and those that did had mixed results as to whether there were any differences in recovery times and outcomes. The other aspect that should be addressed in future studies is the appropriate choice of outcome measures. In the studies analysed in this review, there appears to be no evaluation of how the selected outcome measures match the purpose of the device.
Therapeutic devices such as implants, prosthetic limbs and orthotics offer the promise of revolutionising the medical industry because of their ability to be custom made for the patient, but most therapeutic devices are still early stage, with little research into their efficacy and effectiveness in a clinical setting. There were not enough therapeutic devices tested against a control group to find and evaluate repeated measures of success.
Most studies reviewed were poor quality, meaning that their study designs demonstrated levels of evidence for the 3D-printed device of 4 or 5, with no use of a control group. The 41.4% of papers that reported on individual case studies or provided only qualitative results did not clinically validate the devices that they reported on and therefore add little value to their fields. Few included follow-up or addressed the external validity, bias or confounding factors (see online supplementary material for full comparison of studies). The vast majority of papers reported positive outcomes for the patients, but few studies demonstrated clinically significant findings. Even the level 1 studies demonstrated limited external validity and rarely blinded participants or those measuring the outcomes to the intervention.
The RCTs all demonstrated statistically significant results in favour of using the 3D-printed device over the current clinical practice to which it was compared. However, one RCT23 concluded that the improvement with 3D printing was too small to provide a clinical advantage, and none performed an analysis on the clinical relevance of these differences. The studies consistently used p values of <0.05 as their measure of device success, but taking into consideration the false discovery rate, it is likely that this will only show a real effect about 70% of the time under ideal testing conditions.24 While the critical analysis in table 3 shows all level 1 studies as adequately powered, this is because the Quality Index bases its definition of ‘adequately powered’ on whether a difference due to chance is less than 5%. Therefore, given that all level 1 studies had 40 or fewer participants in each group, many of these studies are likely to be underpowered.24 Considering the possibility of underpowered studies, the publication bias towards positive trials and the fact that all RCTs in this review were single-site trials, the likelihood that the results shown in these papers reflect a real effect is lower than suggested by the reviewed studies.25 Until independent groups start validating 3D-printed devices, it is also difficult to avoid researcher bias.
Most 3D-printed device studies that were excluded from this review had uses in medical training or as moulds for manufacturing and therefore did not meet the inclusion requirements. A few other studies were excluded because they had not been translated into English.
Previous reviews have documented the uses of 3D printing for developing patient-specific medical devices. However, there has been limited research into assessing the efficacy and effectiveness of these devices. This rigorous systematic review design is the first to compare 3D-printed devices across all medical fields and assess their efficacy and effectiveness, describing key benefits that have been found from using 3D-printed devices clinically. The progress made in different medical fields is compared using validated quality assessment and clinical level of evidence tools. It demonstrates that the fields of oral and maxillofacial surgery and the musculoskeletal system are leading the way in validating 3D-printed devices for clinical use. Multiple high-quality studies have been performed on surgical guides for maxillofacial, hip and knee surgeries. This growing body of comparable high-quality research sets an example for other fields to emulate in order to demonstrate the efficacy and effectiveness required to integrate these 3D-printed devices into clinical practice.
A critical appraisal of the efficacy and effectiveness of 3D-printed devices across medical fields provides clinicians with an evidence-based approach to determine the applicability of 3D printing within their field. It also gives researchers an overview of areas that require further research and validation. It encourages investigators to discover what methods have been successfully validated in other medical fields and promotes potential collaborations between fields.
Three-dimensional printing provides a way of customising devices to improve patient outcomes. Available techniques and materials will increase, as 3D printing technology continues to be developed. There is therefore a growing need for validation of new devices, materials and techniques to ensure best patient outcomes. Much research has already gone into developing 3D-printed devices for medical purposes. However, this drive for new technology development has not yet been matched with a drive for critical appraisal of the devices to demonstrate their efficacy and effectiveness. Even the fields that are leading the way in critically evaluating new 3D-printed devices will be required to increase their output as 3D printing continues to grow in popularity and functionality.
The early research covered by this review shows that 3D printing can be valuable for use in medicine. The next important step to take is growing the body of research that focuses on validating 3D-printed devices. All fields require more rigorous and long-term assessments into the efficacy, effectiveness and safety of 3D-printed devices before they are introduced into standard clinical practice. The study is limited to a critical appraisal of individual studies, rather than a meta-analysis, because of the breadth of uses for anatomical models, surgical guides and therapeutic devices and the lack of comparable hypotheses. Funders can take an active role in promoting not only early technological development but also the subsequent clinical trials. Demonstration of clinical efficacy, effectiveness and device safety will become increasingly important as higher risk 3D-printed devices are developed and unconventional manufacturers, such as hospitals and clinics, incorporate 3D printing of patient-specific medical devices into standard clinical practice.
Fields such as oral and maxillofacial surgery, who were early in the uptake of 3D printing, are beginning to stabilise, with a steady number of studies being published each year, but other fields, such as musculoskeletal and circulatory systems, are more recently gaining traction, with increasing numbers of studies performed each year. 3D-printed drug delivery devices and biological 3D printing technologies for printing tissue show huge promise but have not been clinically trialled and so are not included here.26 This is likely to change within the near future. It is therefore recommended that, in this fast-growing and dynamic environment, this review is updated every few years.
Conclusion
This review demonstrates that 3D printing is already being used to develop a broad range of medical devices with clinically effective results. The medical fields of oral and maxillofacial surgery and the musculoskeletal system are leading the way in validating the efficacy and effectiveness of 3D-printed devices and have found that 3D-printed anatomical models and surgical guides are reducing operating times and increasing surgical accuracy. However, the efficacy and effectiveness of 3D-printed devices remains undetermined for the majority of medical fields. 3D-printed devices can have an important role to play in healthcare, but more rigorous and long-term assessments are needed to determine if 3D-printed devices are clinically relevant before these devices can become part of standard clinical practice.
Supplementary material 1
Supplementary material 2
References
Footnotes
Contributors LED, JHMB and MST designed the review. LED and JHMB collected and analysed the data with input from MST. LED drafted the manuscript. All authors contributed to subsequent drafts and approved the final version of the manuscript.
Funding The researchers are financially supported by the General Sir John Monash Foundation and the Wellcome Trust (103383/B/13/Z).
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement All additional data are available in the supplementary material.