Article Text
PDF
Abstract
Context Variability in 2D ultrasound (US) is related to the acquisition of planes of reference and the positioning of callipers and could be reduced in combining US volume acquisitions and anatomical structures recognition.
Objectives The primary objective is to assess the consistency between 3D measurements (automated and manual) extracted from a fetal US volume with standard 2D US measurements (I). Secondary objectives are to evaluate the feasibility of the use of software to obtain automated measurements of the fetal head, abdomen and femur from US acquisitions (II) and to assess the impact of automation on intraobserver and interobserver reproducibility (III).
Methods and analysis 225 fetuses will be measured at 16–30 weeks of gestation. For each fetus, six volumes (two for head, abdomen and thigh, respectively) will be prospectively acquired after performing standard 2D biometry measurements (head and abdominal circumference, femoral length). Each volume will be processed later by both a software and an operator to extract the reference planes and to perform the corresponding measurements. The different sets of measurements will be compared using Bland-Altman plots to assess the agreement between the different processes (I). The feasibility of using the software in clinical practice will be assessed through the failure rate of processing and the score of quality of measurements (II). Interclass correlation coefficients will be used to evaluate the intraobserver and interobserver reproducibility (III).
Ethics and dissemination The study and related consent forms were approved by an institutional review board (CPP SUD-EST 3) on 2 October 2018, under reference number 2018–033 B. The study has been registered in https://clinicaltrials.gov registry on 23 January 2019, under the number NCT03812471. This study will enable an improved understanding and dissemination of the potential benefits of 3D automated measurements and is a prerequisite for the design of intention to treat randomised studies assessing their impact.
Trial registration number NCT03812471; Pre-results.
- ultrasound
- fetal biometry
- reproducibility
- artificial intelligence
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
First prospective assessment of the feasibility of a 3D automated measurement for the complete set of conventional biometric parameters (including abdominal circumference).
Assessment of the potential benefit of automation on ultrasound measurements variability.
Looking further ahead, this research may lead to the rollout of 3D automated measurements in standard care procedures.
The evaluation was carried out as part of a preliminary study. Need to complete the data with a multicentric study on a more significant number of patients.
Introduction
Biometry measurements obtained from two-dimensional 2D ultrasound (2D US) images are part of the assessment of fetal development (standard care). A typical biometric dataset includes measurements of the head circumference (HC), the biparietal diameter (BPD), the abdominal circumference (AC) and the femoral length (FL). Recent studies show that the quality and reproducibility of these measurements are user-dependent1 2 and that the variability of two repeated measurements of the same parameter is probably underestimated.3
In 2D US, image processing techniques and artificial intelligence have been proposed as a method to optimise fetal biometry, and several strategies have been developed.4–7 These strategies can be classified as semi-automated methods. In practice, semi-automated measurement software uses image recognition techniques to select the optimal positioning of measurement callipers in 2D ultrasound. Semi-automated approaches reduce the positioning variability compared with user calliper placement. However, the acquisition of the 2D plane remains user-dependent.
Three-dimensional (3D) US allows the user to obtain the desired reference planes from volumes which have been stored during the examination. The benefit for nonexperienced users is significant, especially regarding the measurements reproducibility,8 9 as the difficulty of finding the correct 2D viewing plane during the exam is removed. In 3D US, automated approaches using artificial intelligence have been proposed, among others, for head structures detection,10 for head planes detection and head measurement11 12 and for long bones measurement.13 14 However, it is more challenging to meet quality criteria for the measurement of the fetal abdomen (AC),1 although it is critical for growth anomaly screening. The aim of the EPICEA project (Optimization of Fetal Biometry with 3D Ultrasound and Image Recognition) is to automatically extract the viewing planes from 3D US acquisitions and to automatically perform biometric measurements, including AC measurements.
Objectives and hypotheses
Primary hypothesis
We hypothesise that the use of automatic plane extraction from volumes reduce the risk of random error during the fetal measurement process.
Primary objective
To assess the consistency between 3D automated and manual measurements extracted from a fetal volume and standard 2D US measurements.
Secondary objectives
To evaluate software (BabySize3D) that produces 3D automated measurements of the fetal head, femur and abdomen from US acquisitions.
To assess the impact of automation on intra-observer reproducibility.
To assess the impact of automation on inter-observer reproducibility (ancillary study).
Clinical investigation design
The optimisation of fetal biometry with 3D US and image recognition protocol (EPICEA – Etude Pilote Croissance Echographie intelligence Artificielle) is a monocentric (Maternité Régionale du CHRU de Nancy, MRUN, France) prospective cross-sectional study. Philips Research is the sponsor and the main financing body of the project. The present version of the protocol (2.0) has been approved by an IRB (Institutional Review Board) on 12 April 2018 and followed the specific guidelines for reporting clinical trials (SPIRIT checklist). Recruitment period (24 months) began in June 2018. The first enrolment was delayed by 4 months due to legislative changes regarding data storage. Recruitment period should end in June 2020.
This clinical investigation is designed as interventional research with minimal risks and constraints (category 2 as defined by Article L.1121–1 of the Public Health Code in FRANCE). We aim to collect standard of care images, prospective volumes and corresponding measurements that are needed for the development and the validation of the investigational software.
Subjects with both all of the inclusion criteria and none of the exclusion criteria and none of the contraindications to investigation procedures are invited to take part in the EPICEA study on a consecutive basis to avoid selection bias.
The primary endpoint of the study is the difference between fetal biometry parameters extracted from automated 3D volume processing, manual 3D volume processing and 2D standard of care measurements.
The secondary endpoints of the study are
The failure rate of the 3D automated measurements.
Examination and postprocessing duration expressed in minutes (ancillary study).
Qualitative comparison of extracted planes from 3D volumes with standard 2D planes.
Interobserver and intraobserver reproducibility by use of intraclass correlation coefficient and Bland-Altman representation (ancillary study).
Devices
The devices to be used within the study are
Commercially available (CE marked) ultrasound equipment EPIQ 5G with commercially available (CE marked) transducer V6-2 and V9-2.
Interactive stand-alone annotation software provided by Philips (nondiagnostic tool) to manually extract the reference plane from volumes
Plane finding and measurement prototype BabySize3D (postprocessing stand-alone software prototype considered as an investigational device, results obtained from this software are not intended to be used for diagnosis). This device is currently set up to perform measurements in the second trimester of pregnancy.
On line electronic case report form (E-CRF), CSOnline 7.5.501.1 Ennov Clinical 2016 to collect the data.
Subjects
Inclusion and exclusion criteria are reported in table 1. All eligible women referred to the US Department of the Maternité du CHRU de Nancy (University hospital) will receive information about this study and explanations by investigators. If the patient is willing to participate consent will be taken before participation. Technical conditions during the US examination and medical history of the patients (including fetus-probe distance, fetal position, placental position, abdominal wall scar tissue, BMI, smoking, diabetes history, high blood pressure history) will be collected in a dedicated E-CRF.
Study inclusion and exclusion criteria
Recruitment procedure
A total of 225 patients will be recruited between 16 and 30 weeks of gestation. Advertising posters will be placed in the waiting rooms of the maternity unit. The poster will mention inclusion criteria, research objectives, specific examinations involved and the research site location.
The enrolment period will be 2 years. The same woman can be enrolled multiple times during her pregnancy, and each enrolment will correspond to new recruitment, with the collection of a new consent form and assignment of a new study ID. Fetal growth and position changes throughout the pregnancy will result in a heterogeneous set of images, even in the case of multiple examinations in the same pregnancy.
Investigators
Investigators in this study (obstetricians and midwives) will be trained to use the ultrasound system in a standardised fashion to secure optimal reproducibility. To limit bias induced by supervised learning15, at least three different operators will contribute to data collection.
Preliminary study
A subgroup of 50 patients was enrolled in the study between 23 October 2018 and 2 May 2019 in order to formalise the acquisition procedure. The data collected were not used for the clinical assessment but contributed to the final development of the BabySize3D prototype. No changes were provided to patients’ standard care.
Clinical procedures
All subjects enrolled in the study will have
Standard US obstetrical examination, including the acquisition of 2D and 3D images and standard biometric measurements (head circumference (HC), biparietal diameter (BPD), abdominal circumference (AC), femur length (FL). The duration will be about 30 min.
After completion of the standard examination, the same operator using the same US system (EPIQ 5G, Philips) will perform the 3D volume acquisitions using V6-2 (October 2018–October 2019) or V9-2 probes (October 2019–October 2020) :
Two head volumes (probe positioning in an axial plan in fetal head direction)
Two abdominal volumes (probe positioning in an axial plan in fetal abdomen direction)
Two thigh volumes (probe positioning in thigh direction in lateral incidence)
Recommendations to investigators regarding volume acquisitions are limited to the global direction of the probe and acquisitions start with no regard to the obtaining of a predefining plan.
The supplementary investigation duration will be about 5 min and will be done immediately after the standard US obstetrical examination and standard biometric measurements. All the data will be reported in a case report form on line (E-CRF).
A subgroup of 50 subjects will be enrolled in an ancillary study and will undergo additional 2D standard measurements and 3D volumes acquisitions done by a second operator as a part of the research activity. As a part of the ancillary study, subjects will undergo:
HC 2D measurement and BPD measurement (to assess interobserver reproducibility in conventional 2D head measurements)
AC 2D measurement (to assess interobserver reproducibility in conventional 2D abdominal measurements)
FL 2D measurement (to assess interobserver reproducibility in conventional 2D femoral measurements)
Acquisition of a head volume (to assess interobserver reproducibility in 3D manual and 3D automated head measurements)
Acquisition of an abdominal volume (to assess interobserver reproducibility in 3D manual and 3D automated abdominal measurements)
Acquisition of a thigh volume (to assess interobserver reproducibility in 3D manual and 3D automated femoral measurements)
The investigation duration for the ancillary study will be about 10 min and the acquisitions will be done immediately after the volume acquisitions of the main study. Figure 1 provides the study design flowchart.
Study design flowchart.
Participation duration of included patients
The study participation starts when the patient signs the informed consent form. The consent is collected after the standard routine US examination by an investigator. Additional consent is collected for the ancillary study. The total scan length is no more than 1 hour. After completion of additional volume acquisitions, the patient participation is considered complete.
Patient participation in the study may be terminated due to (nonexhaustive list):
Investigator decision after the discovery of an exclusion criterion
Patient decision at any time following consent withdrawal
When a patient ends her participation in the study, the investigator reports it in the “end of trial” part of the E-CRF.
Statistical design
A total of 225 patients will be enrolled with the following criteria: complete 2D standard measurements and complete 3D US data sets.
In this sample
A group of 50 patients will be enrolled in order to formalise the acquisition procedure and to contribute to the final development of the investigational device (BabySize3D stand-alone software)
A group of 175 patients will be enrolled for the investigational device evaluation:
A subgroup of 125 patients for the principal study only
A subgroup of 50 patients for both the principal and ancillary study
This sample size estimation is based on the 2D reference measurements at the 50th percentile for AC and FL of the Collège Français d'Échographie Fœtale (CFEF). These measurements are based on the study by Salomon et al.15 In order to show an equivalence between 2D and 3D ultrasound of FL measurements at 22 weeks gestation with an equivalence margin of 1 mm, a mean value of 38 mm (zero difference between the two methods), an SD of 2.42 mm, an alpha risk of 0.05 and a power of 0.95; 153 patients should be included. To take account of measurement difficulties and incomplete data; we increased this number by 15%. A total of 175 patients will be included in the software evaluation.
Stored volumes will be retrospectively analysed. The optimal viewing plane will be identified from the 3D volume and measurements from the calliper placement.
The analysis includes
3D automated processing with BabySize3D stand-alone software (3D automated measurements)
3D manual measurements by one of the investigators. Operators will perform measurements blinded to previous results.
Fifty plans obtained by Babysize3D, the clinical specialist and the standard 2D planes will be reviewed and scored by another operator. This process will assess quality criteria and validate biometric measurements. Rates of unacceptable planes (scoring ≤1) and suboptimal planes (1<scoring ≤ 3) will be estimated. A previously published image scoring method (table 2) will be used for the reviewing process.16
Criteria for score based objective evaluation (Salomon et al 16)
All analyses will be performed with R software (R Foundation for Statistical Computing, Vienna, Austria). The two-tailed significance level will be set to p<0.05. Normally distributed continuous variables will be expressed as mean and SD, non-normally distributed data in medians and IQR and categorical variables as frequencies (percentages).
Primary analysis
Equivalence tests comparing 3D automated measurements with 3D manual measurements and 2D standard measurements (reference) will be computed for each parameter.
Secondary analysis
Mean, SD, median, maximum and minimum for quantitative data. Qualitative data will be described by frequency and percentage.
Mean comparison between parameter measurements or scores will be performed by using paired Student’s t test or Mann-Whitney test according to their type. Reproducibility (intraobserver for the main study and interobserver for the ancillary study) will be evaluated by calculating the intraclass correlation coefficient with its 95% CI. This approach will be completed by a graphical representation using a Bland-Altman plot.
Procedures considering all data
Statistical analysis will be performed on all collected data except the data from the 50 patients which will be used for the BabySize3D software development.
Treatment of missing data
Every effort will be made to complete missing data and remaining missing data will not be reconstituted. There will be no interim analysis.
Termination of study on statistical grounds
Premature termination of this clinical study may only occur because of a regulatory authority decision or at the discretion of the Sponsor (see Section 16 Suspension or Premature Termination), and not on statistical grounds.
The study timeline and SPIRIT schedule of enrolment, interventions and assessments are reported in figure 2 and table 3, respectively.
Spirit schedule of enrolment, interventions, and assessments.
Estimated study timeline.
Data management procedures
The data local storage will be performed via ArchiMed database; this platform is declared to the French authority CNIL (Commission Nationale de l’Informatique et des Libertés No. 1410005).
A data transfer to Philips will be done by the end of this study. For this purpose, an identification code (UID) will be associated with each exported case (2D measurements+2D and 3D acquisitions+rounded Gestational Age). Only authorised individuals associated with the study (investigator, sponsor, health professionals working for the study and quality controllers) will be given access to the data.
All anonymised data will be kept at the investigation site responsible for conducting the study, for preventing loss from computer failure.
The data collected in the E-CRF will be fully anonymised. There will be a periodic (5 years.) review to determine if the data are still useful, that is, needed for the main publication or for the completion of an unpublished study. Data will be deleted if no longer useful.
The sponsor should retain all sponsor-specific essential documents in conformance with the applicable regulatory requirement(s) of the country. If the sponsor discontinues the study, all sponsor-specific essential documents should be maintained for at least 5 years after formal discontinuation or in conformance with the applicable regulatory requirement(s). The sponsor should inform the principal investigator or the investigation site in writing of the need for record retention and should notify in writing when the trial-related records are no longer needed.
Monitoring plan
Monitoring activities will be taken care of by the sponsor according to the ISO14155 standard “Clinical Investigation of medical device for human subjects”. Monitoring procedures will be performed every 50 inclusions.
The investigation site will be responsible for
Exercising reasonable efforts to enrol the number of patients defined in this study. The investigation site may need to screen more than the defined number of patients in order to enrol the number of defined patients;
Ensuring that informed consent is properly obtained from enrolled patients
Maintaining records of enrolment for the study and any other records necessary per institutional requirements;
Developing and maintaining data management tools;
Ensuring that the study is conducted in accordance with the signed agreement and this clinical investigation plan, in particular performing image acquisition and measurements as defined in the aforementioned protocol;
Collecting, assessing, reporting, and managing solicited and spontaneously reported adverse events (AE) and other unintended effects of trial interventions or trial conduct;
Data sharing:
- Anonymised images (US acquisitions) will be shared between the investigation site and Philips Research department. Sharing of data is the responsibility of the investigation site. The investigator’s responsibility includes de-identification.
- Results of all measurements and analysis will be entered by the investigation site in worksheets
- Updated worksheets will be submitted to the sponsor on request
- All patient data will be removed from the images prior to sharing the data in this study
Statistical analysis and summary of the study results
These responsibilities fall to the investigators under the principal investigator (GA) responsibility, to the project manager (CB), to the mandated biostatistician (GH), to technical CHRU staff (regarding ArchiMed database).
The sponsor will be responsible for
Ensuring all the necessary ethic review(s) and approval(s) are obtained
Quality control and enrolment approval (studies with incomplete acquisitions, corrupt files or missing images or measurements will not be approved)
Providing the principal investigator with the necessary information to conduct the clinical study
Preparing and submitting the clinical study application to the Ethical Committee.
Ensuring that any reviewing ethics board and regulatory agencies are promptly informed of any significant new information in the current study
Ensuring compliance with labelling, reporting and record-keeping requirements
Data sharing logistics with Philips Research and the investigation site
Ensuring that the clinical study is conducted in accordance with good clinical practice.
These responsibilities fall to Philips Research Development (CC, CR and LR). A Clinical Research Assistant from Philips will be appointed to conduct the study monitoring.
Ethics and dissemination
Ethical permission
The clinical investigation will be conducted according to the principles of the Declaration of Helsinki and its revisions, the European regulation (UE) n° 536/2014 from the European Parliament related to clinical trials and human medication abrogating the European guideline 2001/20/CE, the law No. 2004–806 of 9 August 2004 on public health policy, the law No. 2004–800 of 6 August 2004 on bioethics, the modified law No. 78–17 of 6 January 1978 on information technologies, files and civil liberties, the law No. 2012–300 of 5 March 2012 on research involving the human person; the law No. 2016–41 of 26 January 2016 on the modernization of the French healthcare system and ordinance No. 2016–800 of 16 June 2016 on research involving the human person and their implementing decrees.
In accordance with Article L. 1123–6 of the Public Health Code, the research protocol has been submitted by the sponsor to the Institutional Review Board (CPP, Comité de Protection des Personnes). The study and related consent forms were approved by CPP SUD-EST three on October 2018, 2nd with reference number 2018–033 B. The study has also been registered in the clinicaltrials.gov registry under the number NCT03812471.
Protocol amendment
Changes to the protocol will be made only by written amendment agreed on by the sponsor and the principal investigator. The ethical committee will be informed of all changes and must approve all changes, and the principal investigator will be notified after ethical committee approval.
Data curation
Data collected for the purpose of the study will be stored in Archimed database. The storage will be renewed every 5 years until the complete analyse and publication of results.
Dissemination
This study will provide data on the relevance of using 3D automated measurements in standard care (feasibility, reproducibility and agreement with conventional 2D measurements). In the short term, these results will be distributed in the form of an original article. In a medium term, these results will enable the design of intention to treat randomised studies assessing the real impact on fetal screening of using artificial intelligence in reducing variability and inaccuracy in fetal measurements.
Risk and benefits assessment
Potential risk related to the study
The risks associated with the investigational device and the clinical procedure applied in this clinical investigation are estimated in accordance with ISO 14971; 2012 titled “Medical devices - Application of risk management to medical devices”.
To avoid any risk of interference with the patient care management, analysis with the investigational device will be performed after deidentification, and the results will not be communicated to the clinical teams in charge of the patients monitoring, and follow-up and all regulatory and ethical guidelines will be followed.
Anticipated clinical benefits
The goal of the study is to evaluate the feasibility of a fully 3D automated analysis of fetal US volumes to extract measurements of abdominal circumference, femur length, head circumference and biparietal diameter. This technique could have a positive impact on reproducibility and quality control of the measurements, especially for less experienced operators. Any improvement in this field could lead to a reduction in complications linked to growth anomalies (intra-uterine death, obstetrical complications) and associated iatrogenic effects (induced premature birth, inadequate obstetric care).
The risks of the study are minimal. No additional risks, in terms of health, can be linked to the participation in this study. Privacy risks will be mitigated, and the benefits of the use of the investigational device outweigh the risks.
Final research report
The principal investigator and the mandated biostatistician will collaboratively write the final research report. This report will be submitted to each of the investigators for review. Once a consensus has been reached, the final version must be endorsed with the signature of each of the investigators and sent to the sponsor as early as possible after the effective end of the research.
Patient and public involvement
We did not involve patients or the public in the design, the conduct or the reporting of our research.
Discussion
This study aims at evaluating the impact of full automation of standard measurements for fetal growth monitoring and fetal size estimation.
The first limitation of this study is related to the skill level of the operators that will participate in data collection. Automated fetal biometry is at a development stage, and additional data is required to train the system to reach human-level performance. The first 50 data sets to be collected will be used for machine learning. This supervised learning approach requires high-quality data.17 These scans will be performed by expert sonographers to ensure that the data is of sufficient quality. For the second data set, the evaluation of the biometry measurements reconstructed from 3D acquisitions will be based on comparison with the 2D acquisitions, thus reflecting expert-level performance.17 Due to this dual methodological constraint (high-level 2D quality data for assessment, and relevant measurements in 3D associated with the acquired data), it was decided to have this second set of acquisitions performed by expert sonographers as well. This choice was also strengthened by the need for global consistency in comparisons between dataset and for ethical issues (to reduce the number of operators interacting with patients, we excluded the possibility of adding non-expert investigators). However, to limit the overfitting bias17 related to expert acquisition and variability reduction into the sample, at least three different experts investigators will perform the 3D acquisitions and the range of gestational age at the time of acquisition is extended from 16 to 30 WG with minimal restrictions on exclusion criteria (eg, coauthors choose not to exclude fetus with growth abnormality) », those aspects guarantee a realistic variability in the data set.
Clinical evaluation of automated fetal biometry will require further study. In order to allow for an optimal evaluation, these studies will have to be performed by operators that are not involved in the software design. A multicentre study may reduce bias in selection bias or inconsistency introduced by clinical practice that deviates from published guidelines.
The main goal of this study is to propose a novel approach to fetal biometry that leverages the advantages of fetal US (availability of ultrasound systems, real-time examination) while trying to reduce its drawbacks (operator dependent measurements, data quality, processes variations).
Through the use of artificial intelligence, larger quantities of data could be processed, as well as more complex information. These innovations might lead to significant changes in the use of fetal US by providing a more accessible and more consistent method to perform measurements. There may be potential to refine the parameters evaluated to understand fetal growth and building new measurement processes.
Acknowledgments
The authors would like to thank the investigators, coordinators, and patients for their involvement in the study. We also thank our colleague Dr Brian Dromey, from UCL Institute for Women’s Health for the manuscript final proofreading.
Footnotes
Contributors GAG (principal investigator) and GH (biostatistician) designed the study; CC-V, CR, and LR have contributed to the software development, GAG and OM wrote the protocol; CB and MB conduct the project, GAG performs and supervises the inclusions, and GH is intended to perform the statistical analysis. All the authors reviewed and contributed to the protocol.
Funding This study was funded by Philips Research.
Competing interests The authors CC-V, CR, LR, are part of Philips Research Development. Philips Research is the sponsor of the EPICEA study. The other authors (GAG, GH, CB, OM, MB) are part of CHRU Nancy and have been mandated by Philips Research to design, to set up and to carry out this research.
Patient consent for publication Publication of data is subject to participant no-opposition (mentioned in information form).
Ethics approval The study was conducted in accordance with ethical principles of the Declaration of Helsinki and Good Clinical Practice guidelines. Written informed consent was obtained from the human participants involved in the study. The research protocol has been submitted by the sponsor to the Institutional Review Board (CPP, Comité de Protection des Personnes). The study and related consent forms have been approved by CPP SUD-EST 3 on October 2018, 2nd with reference number 2018-033B.
Provenance and peer review Not commissioned; externally peer reviewed.