Article Text
Abstract
Objectives Self-Examination Low-Cost Full-Field Optical Coherence Tomography (SELFF-OCT) is a novel OCT technology that was specifically designed for home monitoring of neovascular age-related macular degeneration (AMD). First clinical findings have been reported before. This trial investigates an improved prototype for patients with AMD and focusses on device operability and diagnostic accuracy compared with established spectral-domain OCT (SD-OCT).
Design Prospective single-arm diagnostic accuracy study.
Setting Tertiary care centre (University Eye Clinic).
Participants 46 patients with age-related macular degeneration.
Interventions Patients received short training in device handling and then performed multiple self-scans with the SELFF-OCT according to a predefined protocol. Additionally, all eyes were examined with standard SD-OCT, performed by medical personnel. All images were graded by at least 2 masked investigators in a reading centre.
Primary outcome measure Rate of successful self-measurements.
Secondary outcome measures Sensitivity and specificity of SELFF-OCT versus SD-OCT for different biomarkers and necessity for antivascular endothelial growth factor (anti-VEGF) treatment.
Results In 86% of all examined eyes, OCT self-acquisition resulted in interpretable retinal OCT volume scans. In these patients, the sensitivity for detection of anti-VEGF treatment necessity was 0.94 (95% CI 0.79 to 0.99) and specificity 0.95 (95% CI 0.82 to 0.99).
Conclusions SELFF-OCT was used successfully for retinal self-examination in most patients, and it could become a valuable tool for retinal home monitoring in the future. Improvements are in progress to reduce device size and to improve handling, image quality and success rates.
Trial registration number DRKS00013755, CIV-17-12-022384.
- medical retina
- vetreoretinal
- ophthalmology
Data availability statement
No data are available. Unfortunately, we cannot share any data other than the data in this manuscript. This is mainly due to the fact that the privacy section in the informed consent that all participants signed does not permit widespread data sharing. Furthermore, raw data might contain patient-specific information.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
Pilot trial with a new optical coherence tomography device.
Study in patient group representative for future real-life patients.
No longitudinal data were collected.
Introduction
Neovascular age-related macular degeneration (AMD) can be treated effectively with vascular endothelial growth factor (VEGF) inhibitors.1 However, in real-life, patient adherence to anti-VEGF treatment is lacking considerably compared with pivotal randomised clinical trials (RCTs).2 This leads to undertreatment and thus to significantly inferior treatment results compared with RCTs.3 Long distances from the home to the doctor’s site, combined with many follow-up visits, were identified as the main reasons for non-adherence.4 We believe that home monitoring could very effectively address this issue by eliminating or reducing monitoring visits. At the same time, frequent follow-up in a home monitoring setting is expected to be advantageous for earlier relapse detection, resulting in more just-in-time treatment and possibly better overall treatment results.
As of today, optical coherence tomography (OCT) is the only practical technology with sufficient sensitivity and specificity for disease relapse recognition5 6 and is therefore recommended by all major AMD guidelines.7–9 Other methods such as visual self-testing, Amsler grid testing and preferential hyperacuity perimetry (PHP) are all short of sufficient sensitivity.5 6 10–12 We have previously reported on Self-Examination Low-Cost Full-Field OCT (SELFF-OCT).13 The SELFF-OCT was specifically designed to address the two main challenges of home-monitoring: cutting expensive component costs of OCT technology and allowing self-examination by elderly patients. In comparison to the previously published first prototype, we were able to considerably improve image quality by adjustments in the optical components, and to improve self-operability by adjusting the user interface.
This study investigated the sensitivity and specificity for detecting treatment necessity and AMD-specific biomarkers.
Materials and methods
Technical description of the SELFF-OCT prototypes
The technical design of SELFF-OCT has been described before.13 14 In short, SELFF-OCT creates an OCT volume scan of the central retina without the need for a beam scanner and other expensive components such as spectrometers or tunable light sources by using full-field technology. It uses an extended illumination of the retina by a 0.9 mW parallel beam from a superluminescent diode (SLD-340-UHP-To9-PD, Superlum, Cork, Ireland) with a 840 nm wavelength and a 26 nm spectral bandwidth.
Compared with the first prototype, the optical setup was optimised to reduce reflections and provide a more uniform illumination of the retina. Instead of a manual dioptre adjustment, defocus and astigmatism were corrected numerically in the processing.15 Last, the patient interface was improved. Whereas in the previous study, the patient could freely position his head on a gel cushion, now the patient could choose one out of three three-dimensional (3D) printed headrests that was used for more rigid head positioning. Figure 1 shows the prototype with the new patient interface.
The SELFF-OCT prototype recorded a densely sampled volumetric retina scan of 3.0×1.8×1.4 mm with an axial resolution of 12 µm and a lateral resolution of approximately 8 µm depending on the size of the pupil of the eye.
Study outline
For this prospective single-arm diagnostic accuracy study, we consecutively recruited all patients with neovascular age-related macular degeneration meeting inclusion and exclusion criteria who were treated by the main performing physician (CvdB) in our clinic between 23 July 2019 and 12 March 2020, and who voluntarily agreed to participate after informed consent. One eye (usually the eye that presented with the acute problem) was chosen as study eye, the partner eye was also examined if possible. Main inclusion criteria were patients above the age of 18 years diagnosed with neovascular AMD with the necessity for macular examination including OCT. Main exclusion criteria were significant opacities in the optical media, prior administration of mydriatic eye drops on the examination day and ametropia >±3 dioptres. We further excluded patients with decimal visual acuity under 0.1 in the study eye or obvious difficulty in steady head positioning. No further preselection of patients (eg, fixation testing or geographic atrophy assessment) was undertaken in order to minimise inclusion bias. The study reported was an extension study to the previously published SELFF-OCT study.13
Recruiting was planned for 51 patients according to the same sample size calculations of the first SELFF-OCT study. In this paper, we only report on the patients from the extension study with the improved prototype.
Study protocol
After informed consent was obtained, the patients received a complete assessment of best-corrected visual acuity (BCVA) and a complete examination of the anterior eye segment. Afterwards, usage and handling of the SELFF-OCT was explained to patients by the examiner. No further training documents or videos were necessary. The patient chose the most comfortably fitting out of three 3D printed head masks that differed in size, which was then magnetically mounted on the SELFF-OCT device. The mounting mechanism provided horizontal rails on which the mask could be slid freely so that the patient could position his head in the correct location with his study eye looking into the eyepiece. When positioned correctly, the patient could see a small green fixation target. A later measurement of the partner eye was done by shifting the head mask accordingly.
Once properly adjusted, the patient would start the measurement himself by pushing a hand-held trigger button. During measurement, the patient saw the green fixation target superimposed on the red illumination of the retina by the superluminescent diode. The patient had to keep the fixation target centred on the illumination, which could be done via small head movements.
Within one measurement cycle of 60 s, the device first performed one overview scan to locate the exact position of the retinal pigment epithelium (RPE) and then performed 15 consecutive detailed scans around the RPE that lasted 1.3 s each. Throughout the study protocol, the patient performed several of these measurement cycles. The first measurement cycle M0 (training cycle) was used as an introduction into the technology and was performed under supervision of the attending physician. Only during this first training period, the physician could guide the patient into correct head positioning. In contrast to the following measurement cycles, the M0 measurement could be repeated ad libitum, until both the patient and examiner felt certain that the patient was sufficiently experienced with the operation of the device. Overall, this training process lasted about 2–5 min per patient.
Afterwards, the patient performed two entire measurement cycles (M1 and M2) without medical assistance. They could not be repeated. Two entire non-repeatable measurement cycles (M1b and M2b) were also performed with the partner eye, if possible. Whereas the results of the M0 training cycle were not included in further statistical analyses, the M1, M2, M1b and M2b measurements were included into statistical analyses, regardless of quality or performance. All measurements were taken without prior administration of mydriatic eye drops.
After finishing the SELFF-OCT measurements, all patients received dilatating eye drops, a detailed scan by a reference SD-OCT (Heidelberg Spectralis HRA+OCT2, 6×6 mm volume scan with 49 adjacent B-scans, without enhanced depth imaging mode), a colour fundus photography (Zeiss FF450plus) and a complete binocular funduscopic examination. The OCT scan of the reference system was used as a gold standard to test the diagnostic accuracy of the SELFF-OCT and was performed in mydriasis to achieve maximum benchmark quality. Finally, all patients were asked if they were dazzled by the measurement light or experienced any other adverse events.
Image processing and rating
The acquired images were processed by a custom processing script in the software package MatLab (Mathworks, Natick, Massachusetts, USA). This included reconstruction of the OCT images and computational ametropia correction.15 The reconstructed volumes were averaged with a floating average filter of five adjacent slices in one lateral dimension to improve signal-to-noise ratio (SNR). All images were graded for different artefacts (motion artefacts, saturation artefacts, vignetting, blurring and signal strength of the neuroretina) in a standardised manner published before13 by the author MMo. For each patient and eye, the three images with the most favourable artefact rating (ie, the lowest overall artefact score) as well as the SD-OCT reference scan were exported into DICOM format and were sent to the reading centre (Macula Monitor Münster, Augenzentrum am St. Franziskus-Hospital, Münster, Germany).
Data were graded by two junior graders. In case of deviation between the two graders, a final decision was rendered by an additional senior grader (BH). In addition, the senior grader rated every 10th image for quality control. If there were less than three successful measurements available for one eye, the accordingly reduced number was graded. Data for the SELFF-OCT and the SD-OCT devices were graded independently and in a masked setting. For the biomarkers drusen, intraretinal fluid (IRF), subretinal fluid (SRF), pigment epithelium depletion (PED) and intraretinal hyper-reflective foci, a binary yes/no decision indicating whether they were visible in the images or not was made. The anti-VEGF treatment indication was performed based on the visible biomarkers in accordance with the German AMD treatment guideline.9 In short, because no longitudinal data were available, this practically meant presence of SRF or non-degenerative IRF or diffuse retinal thickening. For the anti-VEGF treatment decision, the rater could also select uncertain if he was <90%, but >50% certain that treatment was necessary.
Analysis of the sensitivity and specificity
SD-OCT was used as a gold standard to test the diagnostic accuracy of the investigational device. Other information (BCVA, colour fundus photo, etc) was not taken into account to allow for direct comparison between the two OCT technologies. For the calculation of single biomarker sensitivity and specificity, all volumes were included. For the calculation of the sensitivity and specificity for anti-VEGF treatment decision, only the measurement with the best artefact rating for each eye was used. Also, for this anti-VEGF treatment decision, eyes with an ‘uncertain’ treatment decision in the gold-standard SD-OCT images were excluded from the analysis. Eyes with an ‘uncertain’ treatment decision based on the SELFF-OCT images were counted as ‘yes’ (treatment indicated), because in home monitoring, an uncertain treatment decision should result in an additional investigation using the clinical gold-standard modality.
Statistics
Differences in success rate between study and partner eye and between phakic and pseudophakic eyes were calculated with a χ2 test in R.16 Differences in patient age and BCVA between successful and unsuccessful eyes was calculated with a Mann-Whitney-Wilcoxon U test. Sensitivity and specificity analyses were performed in a custom MatLab script and double-checked in R. CIs were calculated using a two-sided Clopper-Pearson interval. Inter-rater agreement was calculated using Cohen’s kappa.
Patient and public involvement
Frequent follow-up visits are part of the high treatment burden for patients with AMD that lead to therapy non-adherence. Patients regularly ask for measures to reduce follow-up visits and ways to self-monitor their disease and were the inspiration for this research project. Because of the technical nature of the study, patients were not directly involved in the specific study design.
Results
Study population and success in image self-acquisition
Due to the COVID-19 pandemic at the end of recruitment, we slightly missed the indented target number of 51 patients and ended recruitment after inclusion of 46 patients. One patient who was referred with AMD was excluded from further analysis because after study procedure, diabetic macular oedema was diagnosed instead of AMD. Five partner eyes were not further analysed because their visual acuity was below 0.1 (exclusion criteria). Thus, in total, 85 eyes of 45 patients were analysed.
Patient age was between 57 and 92 years (mean 79). Twenty-two patients were male, 23 were female. Forty-three of all analysed eyes were phakic, 42 pseudophakic. Table 1 shows an overview of all analysed eyes. All patients were able to rapidly understand the SELFF-OCT prototype and to perform self-measurements. All but one patient required a maximum of 2 M0 training cycles (the former needing 4 cycles), mean number of M0 training cycles was 1.38.
After standardised scoring for image quality, as described in the ‘Materials and methods’ section, 74 out of 85 eyes (87%) met the scoring criteria with at least one volume scan and were further processed by the reading centre. Here, one study eye was found to be not sufficient for analysis by the reading centre, resulting in a final success rate to 73 out of 85 eyes (86%, figure 2).
Comparing successful and unsuccessful eyes, no statistically significant correlation was found for BCVA (p=0.11) or age (p=0.87). There was no significant difference in success rate between phakic (88%) and pseudophakic eyes (75%; p=0.12). There was also no significant difference in success rate between study eye (84%) and partner eye (88%; p=0.69). Therefore, further analysis was performed pooled for both study and partner eye. Figure 3 shows patient demographics of all successfully measured eyes. Forty-two of the 45 patients were of the age of 65 years or older, showing that the device could be used in the intended target population.
Figure 4 shows exemplary SELFF-OCT scans in comparison to the representative SD-OCT scan. Both modalities reveal the same structures, although with a smaller field of view and lower SNR in SELFF-OCT. Nonetheless, AMD-relevant biomarkers such as SRF (patient 1 and 2), IRF (patient 3) and PED (patient 1, 2 and 3) are clearly detectable and comparable in their appearance to SD-OCT.
Five participants of the study had already been recruited for assessment with the first prototype. Figure 5 shows a comparison of the image quality between the first and second prototype. In comparison, SNR was improved and contrast between the retinal layers is more clearly defined, leading to better overall image interpretability.
Sensitivity and specificity
Four eyes were graded ‘uncertain’ with respect to anti-VEGF treatment need in SD-OCT. Excluding these images, a sensitivity of 0.94 (95% CI 0.79 to 0.99) and a specificity of 0.95 (95% CI 0.82 to 0.99) for the anti-VEGF treatment decision was found. Sensitivity and specificity analysis was also stratified for detecting the different biomarkers (table 2).
Reproducibility
For three eyes, only one successful SELFF-OCT measurement was available and for 15 eyes only two successful SELFF-OCT measurements were available. For the 70 eyes with at least two available SELFF-OCT measurements, the anti-VEGF treatment decision between the two technically best rated measurements agreed in 68 cases (97%). For the 58 eyes with at least three successful measurements, all three measurements agreed in 57 cases (98%).
Inter-rater agreement for anti-VEGF treatment decision between the two junior raters was substantial and comparable for both SD-OCT (ϰ=0.744) and SELFF-OCT (ϰ=0.736).
Discussion
Advancements in SELFF-OCT technology
Compared with the previously published SELFF-OCT trial, technical advancements in OCT design were integrated into the prototype. Ametropia correction was now performed computationally, sparing the need for optical components and thus minimising both costs and optical reflections. Moreover, it simplified device usage and removed a possible source of error. Also, various other minor changes in the optical components and retina illumination (see ‘Materials and methods’ section) have been integrated. The design of the more rigid 3D printed headrest was able to drastically reduce motion artefacts, even though acquisition time per volume was increased from 0.9 s in the previous version to 1.3 s in the current setup.
Combined, this led to an improvement in the success rate from 0.77 with the old device to 0.86 in the current study. Also, the image quality was improved noticeably (figure 5) and allowed for the high sensitivity and specificity in biomarker detection discussed in this paper. Also, all five patients who underwent examination with both the first and second prototype further univocally found the new user interface to be much more user-friendly.
Suitability for home monitoring and limitations
In this cross-sectional study, we found excellent sensitivity and reproducibility values for SELFF-OCT imaging of AMD biomarkers and detection of anti-VEGF treatment necessity. However, we only evaluated biomarker presence in a binary yes/no fashion, so that this study cannot report on quantitative measurements (biomarker volume) and their reproducibility. Also, we only performed a cross-sectional analysis. Longitudinal studies with actual AMD monitoring will be needed to confirm the study’s findings in the intended use case. Moreover, while the study participant group was representative for patients with AMD in age, it cannot be guaranteed that the future patients could reproduce similar image quality in an at-home setting. However, while image quality could shrinken because the patient could forget correct device operability or could be less motivated, it could also be discussed that due to training, image quality would increase over time. Finally, adherence to frequent at-home OCT testing remains speculative.
Sensitivity and specificity were especially good for detection of SRF. However, while the specificity for IRF detection was equally good, the sensitivity was limited. This was mostly problematic in images with lower SNR (see comparison in figure 6). Besides lower SNR, there are other factors causing this: first, we removed speckle noise in the raw images by applying a volumetric averaging filter. While reliably removing speckle, this caused especially small structures like the edges of intraretinal cysts to be blurred. Second, images acquired by SELFF-OCT also sometimes show projection artefacts in form of a vertical black band-shaped shadow, which cause focally decreased OCT signal that in some cases is hard to differentiate from intraretinal cysts. Examples can be found in patient 1 in figure 4, where two vertical band-shaped shadows appear close to the right edge. These artefacts are caused by inhomogeneities in the tear film or dust in the optical pathway and vary between each volume; therefore, comparison of two or more SELFF-OCT volumes can help with this distinction. Also, motion artefacts in form of horizontal black bands can overshadow IRF (figure 6, yellow arrows).
Outlook
In future studies, we expect an improved sensitivity for IRF detection by increasing the overall SNR and using information from multiple acquired volumes for reducing speckle noise as well as removing vertical projection artefacts. Current lab prototypes already considerably exceed the SNR of the prototype in this study. Optimisations in image postprocessing such as different averaging could also have implications on the IRF detection rate. Moreover, further contrast adjusting could help with IRF detection (figure 6). In a longitudinal home-monitoring, IRF could additionally be detected indirectly via volume monitoring.17 Moreover, more experience in grading SELFF-OCT could improve results.
Furthermore, in a future home monitoring scenario, image interpretation would have to be performed automatically, preferably by artificial intelligence. Recently, it could be shown that with sufficient training, convolutional neural networks (CNNs) outperform human readers in many instances in OCT image interpretation.18 Therefore, CNN-based interpretation of SELFF-OCT could lead to similar and possibly better results than manual grading. We are currently working on CNN-based solutions for automated segmentation and interpretation of SELFF-OCT.19 In the future, techniques such as domain adaptation20 or transfer learning21 of high-quality segmentations created on SD-OCT could be integrated into the training process to further improve the results. In addition, the image quality of SELFF-OCT can be improved, which has a positive effect on interpretability. Also, recent work has shown that CNN-based denoising approaches can lead to good results.22
In this study, we focused on patients with AMD. However, we have shown in our prior study that SELFF-OCT can also detect relevant biomarkers in patients with diabetic macular oedema and retinal vein occlusion, so that home monitoring for these and possibly other diseases could be an option.
Comparison with other monitoring methods
Various other means of AMD self-monitoring are currently in practice, most notably standardised Amsler grid testing, ‘environmental Amsler’ and PHP. The reported diagnostic accuracy of these tests varies widely in literature. In a recent review, Faes et al23 found that sensitivity for detecting wet AMD in a screening setting was 0.34–1.0 (pooled sensitivity 0.78) for Amsler grid testing and 0.68–1.0 (pooled 0.85) for PHP. However, literature on using these methods for anti-VEGF therapy monitoring is limited. A study investigating 70 patients with AMD under anti-VEGF therapy10 found that 81% of the participants used some form of self-monitoring, most notably environmental Amsler and Amsler grid testing. However, sensitivity for subjective recurrence of exudation detection was markedly low with 0.33 (specificity of 0.85) in patients using visual-function self-monitoring, compared with OCT gold standard. Moreover, the sensitivity for subjective recurrence detection was only very slightly elevated compared with the group not performing visual-function self-monitoring, who had a sensitivity of 0.25 and specificity of 0.82. This suggests that these tests add little value in AMD monitoring. Similar findings are also reported in other studies with patients with AMD: in a prospective study with 31 patients, Hoerster et al5 report a sensitivity of 0.15 for subjective patient perception and 0.3 for Amsler testing. In a larger retrospective study with 638 patients, Bruender et al6 found a sensitivity/specificity of 0.28/0.86 for a standardised interview asking for subjective deterioration in visual acuity or metamorphopsia and a sensitivity/specificity of 0.27/0.86 for a ≥1 line loss in standardised visual acuity testing.
For PHP, there is less data on anti-VEGF treatment monitoring. In a small study of 17 patients, Querques et al11 found encouraging results with a sensitivity of 0.83 and specificity of 0.67 for detection of exudation. Other studies, however, failed to replicate these encouraging results: Thomas et al12 only found a sensitivity/specificity of 0.17/0.86 in 35 patients. As for visual acuity and Amsler testing, it can be assumed that morphological changes will precede functional changes.
Comparison with other home monitoring OCT candidates
Recently, other research groups have also investigated possible OCT devices for home monitoring purposes. A SD-OCT device called sparse OCT was investigated in 62 eyes of 31 patients with AMD.24 The device performs a volume scan with a field of view of 3.8×3.8 mm. Due to lower A-scan rate than clinical devices, in volume scans, image quality is also reduced. Self-measurement was successful in 94% of all eyes. The group did not report on any sensitivity in disease activity or biomarker detection.
A commercial SD-OCT device by Notal Vision was tested in 74 eyes of 45 patients.25 Success rate in image acquisition was 93%. Sensitivity and specificity for SRF detection was 0.93/0.96, for IRF detection 0.88/0.98, for any type of fluid 0.89/0.97. Direct comparison between our prototype and the Notal OCT is complicated due to different patient characteristics and study designs. Generally, however, despite the more cost-expensive SD-OCT technology and better IRF sensitivity, overall diagnostic accuracy was comparable to our approach.
Both approaches rely on SD-OCT. Even though efforts have been made to cut down component costs,26 it remains challenging to meet the cost-requirements for a broadly available home monitoring OCT system.
Conclusion
Our study shows that SELFF-OCT is a highly sensitive and specific diagnostic tool for AMD activity detection. It outperforms both subjective visual function tests and prior published device-based visual function tests (PHP) in diagnostic accuracy. Most patients with AMD were able to self-operate the device. SELFF-OCT therefore is a promising candidate for a reliable therapy home monitoring in patients with AMD and possibly other diseases. With the low-cost design, we believe that it can realistically be used in real-life care in the near future. A home-monitoring system that alerts the patient in case of disease activity may increase patient compliance with and adherence to treatment. With early relapse recognition and timely treatment, it further could possibly improve overall therapy outcome.
Data availability statement
No data are available. Unfortunately, we cannot share any data other than the data in this manuscript. This is mainly due to the fact that the privacy section in the informed consent that all participants signed does not permit widespread data sharing. Furthermore, raw data might contain patient-specific information.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by Ethics Committee, University of Kiel (reference number: A139/17). Participants gave informed consent to participate in the study before taking part.
References
Footnotes
Contributors The investigational device and the underlying technology were researched, devised and built by CvdB, HS, MMo, PK, MMü, MvE, TK, DT-K, GH and JR. The study was planned by CvdB, HS, JT, CE, KP, DT-K, IK, GH and JR. All study data were acquired by CvdB. The data were analysed by CvdB, HS, JT, CE, KP, MMo, BH, PK, MMü, TK, DT-K, GH and JR. CB is responsible for the overall content as the guarantor. All of the authors were included in the editing of this paper.
Funding The study is supported by Federal Ministry of Education and Research (BMBF) (Grant numbrer: 13N13766).
Competing interests HS, PK, MMü and GH hold a patent related to SELFF-OCT. None of the other authors has any conflicts of interest to disclose.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.