Article Text
PDF
Abstract
Background/aims To develop a standardised grading scheme, using standard photos, for spectral-domain ocular coherence tomography (SD-OCT) images of the posterior eye and evaluate the interobserver agreement among trained ophthalmologists in identifying pathological changes.
Methods Subjects were recruited from Liwan District, Guangzhou, with SD-OCT data collection from June 2013 to November 2013 as part of 10-year follow-up visits from the Liwan Eye Study. All subjects underwent SD-OCT imaging of the macula with scanning lines analysed by two ophthalmologists to assess for the presence of 12 different posterior segment lesions. Per cent agreement for each lesion between the graders and quantitative measures of dome-shaped macula (DSM) height and choroidal thickness were calculated.
Results A total of 679 SD-OCT images from 679 subjects were independently evaluated by the two graders. Each of the 12 lesions was successfully graded as present or absent in over 96% of images. For all lesions, per cent agreement between observers was over 90%, ranging from 90.7% for epiretinal membranes and retinal pigment epithelium thickenings to 99.7% for full thickness macular holes and retinal detachments. Quantitative measurements of DSM height and choroidal thickness at three locations of the eye all exhibited intraclass correlation scores between the two graders of greater than 0.9.
Conclusion Our study demonstrates high concordance between graders in characterising posterior segment lesions using SD-OCT images, validating the continued use of this imaging modality in the diagnosis of posterior eye disease.
- optical coherence tomography
- posterior segment
- qualitative grading
- quantitative measurement
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The study proposes and validates a set of standardised criteria for grading posterior segment spectral-domain ocular coherence tomography images in accordance with International Nomenclature for Optical Coherence Tomography consensus anatomic landmarks.
The study assessed for 12 different pathological changes of the vitreoretinal interface, neurosensory retina and retinal pigment epithelium/Bruch’s complex and presents standard photos for each.
The study reports interobserver variability between trained ophthalmologists in grading posterior segment lesions, analogous to how these images are interpreted in most medical settings.
Ocular coherence tomography images were obtained from a population-based study, resulting in low prevalence for several of the types of lesions.
Introduction
First introduced in 1991, optical coherence tomography (OCT) is a non-invasive imaging modality based on interferometry, able to generate real-time cross-sectional maps of the retina.1 Analogous to B-scan ultrasound, OCT divides light from a broadband light source into sample and reference beams and measures the interference pattern following backscatter from the back of the eye,2 resulting in a reflectivity versus depth profile of the retina useful in diagnostic decisions. Recently, widespread adoption of spectral-domain OCT (SD-OCT) technology has effectively replaced previous slower time-domain OCT devices, allowing for image acquisition rates of up to 29 000 axial scans per second with a 6 µm resolution or 14 500 scans per second with 3.5 µm resolution.3 With the advent of swept-source OCT using longer bandwidth light sources and higher resolution photodetectors, these numbers will only continue to improve.
Natural variation in reflectivity of normal retinal tissue underlies the basis of OCT to resolve the multiple layers comprising the vitreoretinal interface, neurosensory retina, retinal pigment epithelium (RPE)/Bruch’s complex and inner choroid. While nerve fibres and the RPE demonstrate high reflectivity, plexiform and nuclear layers show medium reflectivity, and photoreceptors display low reflectivity.4 5 In contrast to many subfields of medicine in which a tissue in question can readily be biopsied, ophthalmology and particularly the study of the retina are disproportionately dependent on imaging. Consequently, in clinical practice, OCT has become a routine and invaluable test to identify an extensive array of lesions ranging from epiretinal membranes (ERMs) and macular holes to pathological changes in the RPE.
OCT reading guidelines to date have been proposed for a number of conditions including glaucoma, diabetic maculopathy and age-related macular degeneration,6–8 but a comprehensive OCT classification scheme for the posterior segment changes has not yet been developed. In 2015, Heng et al 9 demonstrated a high percentage concordance between two retinal specialists in evaluating features of diabetic macular oedema. Similarly, in 2016, Kim et al 7 quantified the intergrader agreement among glaucoma specialists of using OCT images in the structural diagnosis of glaucoma. However, for many ocular lesions detectable on OCT, the reliability in grading among the specialists who diagnose them has not yet been characterised.
The purpose of this study was to develop a standardised grading scheme, using standard photos, for SD-OCT images of the posterior eye and to evaluate the interobserver agreement among trained ophthalmologists in identifying pathological changes. Although the applications of OCT in managing ocular disease have been widely celebrated, only recently has consensus terminology for normal posterior segment SD-OCT imaging been proposed,10 and little has been reported on the interobserver variability in assessing posterior segment disease by SD-OCT. Here, we define and investigate the grading of 12 different posterior eye lesions by SD-OCT to determine areas of agreement among the images collected in the Liwan Eye Study, a population-based study.
Methods
Subjects
All subjects were enrolled from Liwan District, Guangzhou, with OCT data collection from June 2013 to November 2013 as part of 10-year follow-up visits from the Liwan Eye Study. The methodology of Liwan Eye Study has been described in detail elsewhere.11 In short, all subjects aged 50 years and older residing in the selected study clusters in Liwan District of Guangzhou for more than 6 months were enrolled in 2003. Written informed consent was obtained from all subjects after explaining the purpose of the study and the risks and benefits of the examination. From these subjects, those who underwent SD-OCT imaging between June 2013 and November 2013 were then included in the present analysis. This research adhered to the Declaration of Helsinki.
Image acquisition
SD-OCT imaging of the macula was performed by a trained technician using high-definition OCT (Model Ivue100, Optovue, Fremont, California, USA) with a retina map scanning protocol. Pupils were not dilated for OCT scannings. Images with a quality score less than 30 as self-evaluated by the OCT were retaken. If low quality of the image persisted despite multiple retakes, the image was excluded from grading. For each graded eye, all scanning lines of the macular were analysed to assess for the presence of lesions. Images of the left eye were taken only if the examination of the right eye was not possible. Database entries were managed with EpiData V.3.0 or greater (EpiData Association, Odense, Denmark).
Classification of lesions
The layers of the macula in physiological conditions were identified on the basis of the classification system proposed by the International Nomenclature for Optical Coherence Tomography (IN•OCT) Panel, which that subdivides the retina into 18 layers progressing from the vitreoretinal interface towards the choroid–scleral interface.10 For each OCT scan, the vitreoretinal interface, neurosensory retina (layers 1–13), RPE/Bruch’s membrane complex (layer 14) and choroid (layers 15–18) were examined. A list of pathological lesions on SD-OCT proposed in this study are presented in table 1. Additional grading details and standard photographs for each lesion are presented in online supplementary table 1 and figures 1–3.
Supplementary file 1
Definitions of assessed SD-OCT lesions
Optical coherence tomography photographs from the Liwan Eye Study showing examples of (A) vitreomacular adhesion, (B) vitreomacular traction and (C) epiretinal membrane.
Optical coherence tomography photographs from the Liwan Eye Study showing examples of (A) full thickness macular hole, (B) lamellar macular hole, (C) macular retinoschisis, (D) retinal detachment, (E) macular oedema and (G) intraretinal hyper-reflective lesion. Figure part F depicts the macular tomographic map for the image displayed in figure part E featuring regions with retinal thickness exceeding 300 µm.
Optical coherence tomography photographs from the Liwan Eye Study showing examples of (A) retinal pigment epithelium thickening, (B) pigment epithelium detachment and (C) dome-shaped macula.
Grading of lesions
Grading criteria and a set of standard photos for all OCT lesions assessed in this study are detailed in online supplementary materials. Briefly, lesions on each image were provided without other clinical information and were independently graded by trained observers into three mutually exclusive categories based on presence or absence of the lesion: ‘None’ (greater than 50% certainty that a lesion is absent), ‘Yes’ (greater than 50% certainty that a lesion if present) or ‘Cannot grade’ (poor image quality or image obscured by mixture of lesions present). Images from each subject were also graded on the ability to measure dome-shaped macula (DSM) height if such a lesion was present and choroidal thickness at three locations along the horizontal scan.
Two ophthalmologists served as observers to grade the OCT images. For qualitative OCT evaluations, interobserver agreement (percent of graded cases agreed between two assessments by different graders) was determined as a measure of inter-rater reliability. For quantitative measures of DSM height and choroidal thickness, descriptive statistics using the Student’s t-test and intraclass correlation (ICC) scores were also calculated. ICC was determined with Stata V.12.0 using a two-way random effects model. For statistical analyses, a P value less than 0.05 was deemed significant.
Results
A total of 679 OCT images from 679 subjects (379 women and 300 men; mean age 70.8±8.1 years) were independently graded by two trained observers. On average, grading required approximately 5 min per image. Each lesion from table 1 was successfully scored as present or absent in over 96% of images with successful grading of all images for retinal detachments (RDs), intraretinal hyper-reflective lesions, pigment epithelium detachments (PEDs) and DSMs.
Table 2 presents the number of cases graded as present or absent by both observers for each lesion, as well as the interobserver agreement for each type of lesion assessed. For all 12 lesions in this study, per cent agreement between observers was over 90%, ranging from 90.7% for ERMs and RPE thickenings to 99.7% for full thickness macular holes (FTMHs) and RDs.
Interobserver grading agreement on qualitative optical coherence tomography evaluations
Quantitative evaluations of DSM lesion height and choroidal thickness at three locations of the eye are shown in table 3. For all quantitative evaluations, the ICC score between the two graders was greater than 0.9, indicating high reliability between graders for these measurements.
Interobserver grading agreement on quantitative OCT evaluations
Discussion
Based on recent definitions proposed for normal anatomic landmarks by the IN•OCT Panel,10 we defined and compared the scoring of 12 types of posterior eye lesions collected from a population of 679 subjects to evaluate the reliability of SD-OCT grading between clinicians. The highest per cent agreement scores were observed in the grading of FTMHs, lamellar macular holes and RDs, all of which are routinely diagnosed and monitored with the assistance of OCT in practice.12–14 Quantitative OCT evaluations also demonstrated a small but statistically significant difference in measurements between graders for DSM height and choroid thickness at nasal, temporal and subfoveal locations. The ability to accurately and non-invasively measure these parameters with OCT is important for both detecting new lesions and monitoring existing ones. In particular, SD-OCT may be of value to track conditions with associated choroidal thickness changes such as pachychoroid neovasculopathy and neovascular age-related macular degeneration.15 16 However, the potential for small discrepancies between independent graders measuring choroidal thickness with this modality should be noted.
From our analyses, the lowest per cent agreement scores were seen with assessments of macular oedema (MO), vitreomacular adhesions, ERMs and RPE thickening. The diversity of disease states leading to MO make it a challenging condition to diagnose, with prior attempts at characterising MO features by SD-OCT exhibiting good but imperfect agreement between independent image readers.9 17 Likewise, several forms of RPE thickening have been reported in the literature, yet the reliability among clinicians in using SD-OCT to identify this type of lesion has not previously been examined. For RPE thickening in particular, our grading criteria were based on examples from the literature of RPE proliferation, hypertrophy, pigmentation and migration changes, all of which notably lack a quantitative definition.18 19 The more subjective definition of RPE thickening on OCT may have made grading this lesion more difficult and may help explain its lower per cent agreement scores relative to other lesions in our study population. Further research should attempt to more precisely define criteria for these lesions to determine the reliability of these diagnoses from SD-OCT with greater confidence. Nonetheless, none of the lesions examined in this study showed per cent agreement between the graders of less than 90%, underscoring the acceptable clinical utility of OCT for evaluating these changes in the eye.
Limitations of this study include the low prevalence in the enrolled population of FTMH, RD, PED and vitreomacular traction (VMT). VMT was not graded as present in any of the 679 subjects, dampening any conclusions that we can draw about this lesion. Limitations regarding the low prevalence of these lesions are to be expected as subjects were drawn from a population-based study. Given the growing adoption of SD-OCT in ophthalmic clinics and its increasing role in patient care, it is important that future studies verify the reliability of OCT image interpretation in daily clinical practice to avoid potential missed or erroneous diagnoses. Fortunately, in practice, the identification of these lesions is complemented by a range of imaging and diagnostic tools including fluorescein angiography, fundus autofluorescence and B-scan ultrasound.20 The collective findings from these tests would inform the results of SD-OCT, improving diagnostic accuracy and the possibility to intervene with appropriate treatment.
In summary, we proposed a standardised grading scheme for 12 different ocular pathological changes of the vitreoretinal interface, neurosensory retina and RPE/Bruch’s complex by SD-OCT using recent IN•OCT consensus anatomic landmarks and assessed the agreement between trained clinicians in identifying these lesions. Our study demonstrates high concordance between graders in characterising SD-OCT images of these regions, providing validation for the continued use of SD-OCT as an ancillary test in the diagnosis of posterior eye pathology.
References
Footnotes
Contributors SKW, XG and MH designed the study, performed statistical analyses and drafted the manuscript. OX, YC, RL and WH collected the data used in the study. All authors read and approved the final manuscript.
Funding MH receives support from the National Natural Science Foundation of China (81125007) and the Fundamental Research Funds of the State Key Laboratory of Ophthalmology, Science and Technology Planning Project of Guangdong Province (92013B20400003). WH receives support from the National Natural Science Foundation of China (81570843H1204). MH receives support from the University of Melbourne at Research Accelerator Program and the CERA Foundation. The Centre for Eye Research Australia receives Operational Infrastructural Support from the Victorian State Government.
Competing interests None declared.
Patient consent Obtained.
Ethics approval This study was approved by the Zhongshan University Ethics Review Board and the Ethics Committee of Zhongshan Ophthalmic Center
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement Original data are available on request. Please contact the corresponding author for further information.