Article Text
Abstract
Aim To investigate whether the publication of the National Institute for Health and Clinical Excellence (NICE) glaucoma guidelines had an effect on the agreement of examination findings between professionals involved in an established glaucoma referral refinement pathway.
Methods To report inter-professional agreement for the clinical examination findings of optometrists with a special interest in glaucoma (OSI), optometrists with no specialist interest in glaucoma (non-OSI) and a glaucoma consultant. Part 1 investigated agreement between an OSI and consultant and part 2 investigated agreement of clinical findings between the non-OSI and a specialist clinician (OSI or consultant).
Results Part 1: Agreement between OSI and consultant in determining an abnormal intraocular pressure (IOP) (>21 mm Hg) expressed as a percentage positive predictive value (PPPV) was no different pre-NICE (60.6%) and post-NICE (61.4%, p=0.51) guidelines. PPPV for identification of an abnormal optic disc was better pre-NICE (60.6%) than post-NICE (42.7%, p=0.02). The appropriate referral rate for patients referred by an OSI was higher pre-NICE (69.6%) than post-NICE (61.2%) (p=0.07). Part 2: The PPPV between non-OSI and specialist clinician for an abnormal IOP was better pre-NICE (62.5%) than post-NICE (50.9%, p=0.12). This was also observed for abnormal optic discs, 70.0% pre-NICE and 52.9% post-NICE (p=0.04).
Conclusions The accuracy for detecting an abnormal IOP by the OSI has remained unchanged post-NICE, but there was a reduction in accuracy in detecting an abnormal optic disc as well as the appropriate referral rate. For the non-OSI, there was a decline in both IOP and optic disc assessment accuracy.
- Glaucoma
Statistics from Altmetric.com
Introduction
The vast majority of referrals to an Ophthalmology department in the UK for suspected glaucoma are initiated by community based optometrists.1 Detection of glaucoma, a progressive optic neuropathy, relies on accurate measurements of the optic disc and also the functional consequences of the disease, measured by visual field (VF) testing. Raised intraocular pressure (IOP) is an important risk factor for glaucoma and hence the importance of also performing this measurement in primary care. Measuring the IOP with Goldmann or Perkins tonometry and slit-lamp binocular indirect ophthalmoscopy of the optic disc are core competencies of an optometrist's training, although in day-to-day clinical practice these examination techniques are often not used by community optometrists as part of a routine eye examination.2 ,3 The published literature also reports a range in the agreement of examination and management between optometrists and ophthalmologists.4–8
Numerous glaucoma referral refinement schemes are operating successfully in the UK, though, being locally commissioned, they have marked differences in organisation set-up and referral criteria.9–16 A diploma in glaucoma or accreditation by a local hospital Ophthalmology department is often a prerequite before an optometrist can undertake glaucoma referral refinement.17–19
The Community and Hospital Allied Network Glaucoma Evaluation Scheme (CHANGES) involves both a triage of the referral letter from the original optometrist with no specialist interest in glaucoma (non-OSI) into a high or low risk category and subsequent clinical refinement of referrals for suspected primary open angle glaucoma. It has demonstrated that community based optometrists with a special interest in glaucoma (OSI) can effectively reduce the number of unnecessary referrals attending the hospital glaucoma service in low risk patients.9 Good agreement between the examination findings of a specialist optometrist in glaucoma and the hospital were reported with this and other schemes before the National Institute for Health and Clinical Excellence (NICE) glaucoma guidelines were published in April 2009.6 ,7 ,9 ,11 ,12 The publication of the NICE guidelines prompted a recommendation from the Association of Optometrists (AOP) to refer all patients with an IOP of greater than 21 mm Hg regardless of the method of measurement.20 ,21
Subsequently, in December 2009, the Royal College of Ophthalmologists and College of Optometrists released the Joint College Guidance (JCG), which was updated in December 2010. It was recommended that a practitioner may consider not referring a patient aged 80 years and over with an IOP of less than 26 mm Hg, or for patients aged 65 years and older with IOP less than 25 mm Hg, if the remainder of the ocular examination is normal.22
The impact of both of these guidelines on glaucoma referral refinement schemes, including agreement in examination findings, has not been reported. The purpose of this study was to assess the effect on agreement in clinical findings between clinicians in the referral pathway since the publication of the NICE guidelines and AOP response.
Methods
CHANGES scheme
Eight OSIs working in the CHANGES scheme cover all referrals from the catchment area of Hinchingbrooke Hospital, Cambridgeshire. Each OSI uses equipment standardised to that used in the hospital glaucoma service, namely a slit-lamp, a Humphrey visual field analyser (Carl Zeiss Meditec, Dublin, California, USA), applanation tonometer (Goldmann model; Haag-Streit, Bern, Switzerland) and digital fundus camera for imaging of the optic disc (Topcon, Tokyo, Japan). Each optometrist received training by the hospital glaucoma team in the form of four half-day sessions with practical examination of the optic disc and the correct use of the Goldmann tonometer. In addition, they had all achieved a nationally recognised postgraduate certificate in glaucoma shared care (City University, London).
Present study organisation
In the CHANGES scheme, an experienced hospital based optometrist with a diploma in glaucoma reads and categorises all referral letters for suspected glaucoma into either low or high risk according to a protocol based on the examination findings listed in the letter. A referral is deemed low risk if only none or one of the following risk factors were noted: abnormal optic disc, abnormal visual field and abnormal IOP (22–28 mm Hg or IOP asymmetry of more than 5 mm Hg). All other referrals are deemed high risk (including any reference to a shallow anterior chamber or IOP >28 mm Hg). Low risk patients are seen by an OSI of their choice within their community practice and high risk patients are seen directly in the hospital glaucoma clinic (figure 1). IOP, optic disc and VF findings are recorded by all OSIs. One in four patients who were discharged after the OSI assessment were sampled and offered a hospital appointment where they were reviewed clinically by the consultant as a safety measure during the first 6 months of the scheme. A patient is discharged if he or she is found to have all the following features in both eyes: an IOP below 22 mm Hg, a normal optic disc after dilated optic disc examination, normal visual fields and Van Herick's temporal limbal chamber depth deeper than 15% of corneal thickness.
Outcomes measured
An IOP measurement of greater than 21 mm Hg in either eye was considered ‘abnormal’ IOP and an optic disc that had an appearance that was considered suspicious for glaucoma was considered an abnormal optic disc.
Inter-professional agreement for components of the examination was assessed for (1) OSI and consultant (Part 1) and (2) non-OSI and a specialist clinician, be it OSI or consultant (Part 2).
Part 1: Data from all new patients seen by the OSI since the scheme started in August 2006 until June 2011 were analysed. The pre-NICE and post-NICE agreement between the OSI and one of two consultant ophthalmologists (fellowship-trained in glaucoma, RB and LC) for the identification of an abnormal IOP and optic disc was undertaken. The analysis consisted of all referred patients by the OSI as well as a sample of discharged patients in order to calculate the appropriate referral rate as well as an estimate of the inappropriate referral rate.
An appropriate referral is defined as one where the consultant agreed with the decision to refer to the hospital on account of one or more abnormal glaucoma examination findings.
Part 2: The examination data from the referral forms of non-OSIs were collected for all new patients in four 2-month time periods: pre-NICE (March and April 2009), post-NICE (November and December 2009), post-original JCG (August and September 2010) and post-updated JCG (March and April 2011). These data were collected as part of the Health Innovation & Education Cluster Glaucoma Pathways project.23
The agreement between the gold standard professional (consultant ophthalmologist for part 1 and either the OSI or the consultant ophthalmologist for part 2) was expressed as a percentage positive predictive value (PPPV) of the referring clinician's findings and represents the proportion of referrals that were true positives.
Data management and statistical analysis
Data from electronic patient records and paper copies of referral letters were collated on Microsoft Excel; statistical analysis was performed in R (V.2.15.1, R foundation for statistical computing).
Fisher's exact test was used to test for statistical significance, as the samples were non-matched.
Results
Part 1
Agreement between the OSI and the consultant ophthalmologist specialising in glaucoma.
A total of 850 patients with low risk referral letters were sent an appointment to see the OSI, of whom 760 attended (non-attendance rate 10.6%). The average age of attendees was 59.5 years and 46% were male subjects.
In the 32 months from the scheme's introduction to the publication of the NICE guidelines, a total of 277 patients were seen by an OSI (8.7 per month), of whom 184 (66.4%) were referred to the hospital and 93 (33.6%) discharged.
In the 19 months post-NICE, 483 patients were seen (25.4 per month), of whom 289 (59.8%) were referred to the hospital and 194 (40.2%) discharged.
Agreement for abnormal IOP assessment
The PPPV for correct identification of an abnormal (>21 mm Hg) IOP in either eye was 60.6% pre-NICE compared with 61.4% post-NICE (p=0.51, Fisher's exact test).
Agreement for abnormal optic disc assessment
The PPPV for the correct identification of an abnormal optic disc was 60.6% pre-NICE compared with 42.7% post-NICE (p=0.02, Fisher's exact test).
Agreement for outcome of OSI appointment
The appropriate referral rate for all patients referred by an OSI to the hospital was 69.6% pre-NICE and 61.2% post-NICE (p=0.07, Fisher's exact test).
Overall, 92% (11/12) of patients who were sampled and clinically reviewed by the consultant after a normal OSI assessment were confirmed as being normal and discharged after this hospital visit, with one patient being retained for further follow-up and given a diagnosis of glaucoma suspect. There were no cases where the consultant diagnosed glaucoma through this sampling process.
Part 2
Agreement between the non-OSI with a clinician with specialist glaucoma training (either the OSI or the consultant ophthalmologist).
A total of 434 patients were referred by the non-OSI during the four 2-month periods of data collection, with an average age of 62.1 years and 42% were male subjects. A total of 304 of these patients x(39% male subjects) were deemed to be high risk, with an average age of 64.0 years. The remaining 130 patients were classified as low risk (43.8% male subjects), with an average age of 57.4 years. Patients with high risk referrals were significantly older than low risk (p=0.01, independent t test), while there was no significant gender difference.
Agreement for IOP assessment
The non-OSI had documented the IOP in 87.6% (380/434) of referral letters for suspected glaucoma (90.0% in the low risk group and to 86.5% in the high risk group), with the use of Goldmann applanation tonometry documented in 46 (12.1%) of referrals and ‘air puff’ non-contact tonometry or no documentation of method of tonometry in the remainder.
The PPPV for correct identification of an abnormal IOP in either eye was 62.5% pre-NICE compared with 50.9% post-NICE (p=0.12, Fisher's exact test).
Where the non-OSI noted an IOP between 22 and 28 mm Hg, the OSI agreed that the IOP was abnormal in 49.4% (40/81). Where the non-OSI noted an IOP greater than 21 mm Hg in conjunction with another examination finding suspicious of glaucoma or where the only abnormal finding was an IOP greater than 28 mm Hg, the consultant noted an IOP greater than 21 mm Hg in 56.0% (28/50). This increased to 75.0% when the non-OSI found the IOP to be greater than 28 mm Hg.
A temporal trend was observed where low risk and to a lesser extent high risk groups demonstrated a decline in the PPPV more recently, though an increase of PPPV was noted post-updated JCG (these are detailed in table 1).
Agreement for optic disc assessment
The non-OSI had documented the optic disc appearance in 78.8% (342/434) of all referral letters. Among the letters triaged as ‘low risk’, 83.4% had optic disc appearance documented, while this was noted in 76.6% of high risk letters.
The PPPV for the correct identification of an abnormal optic disc was 70.0% pre-NICE compared with 52.9% post-NICE (p=0.04, Fisher's exact test).
A temporal trend was observed in which the low risk group demonstrated a larger reduction in PPPV post-NICE than the high risk, with both groups improving post-original JCG and declining once more post-updated JCG (these are detailed in table 1).
Discussion
A dramatic increase in the number of glaucoma referrals has occurred since the introduction of the NICE guidelines, which has placed an increased burden on ophthalmology out-patient departments nationally.24 This is reflected in the observation of increased scheme activity from a mean of 8.7 patients per month pre-NICE to 25.4 per month post-NICE, coupled with an increased discharge rate from 33.6% to 40.2%. These changes in referral activity suggest that the NICE guidelines and, more specifically, the response of the AOP to these guidelines has resulted in an increase in the numbers being referred by non-OSIs for suspected glaucoma coupled with a reduction in the diagnostic accuracy of these referrals. The reduction in the appropriate referral rate from an OSI to the hospital (from 69.6% pre-NICE to 61.2% post-NICE) would appear to be a consequence of this change in referral activity.
There was no change in the PPPV for the identification of an abnormal IOP between an OSI and a consultant in the post-NICE period compared with pre-NICE, explained by the standardisation of techniques employed by both the OSI and the consultant to measure the IOP. It is has been reported that in the period post-NICE guideline introduction, the total number of referrals for only raised IOP and the unnecessary referral rate associated with these referrals has increased.24 ,25 Our results indicate that the probability of a consultant reproducing the non-OSIs finding of a moderately raised IOP (22–28 mm Hg) is lower than that of a significantly raised IOP (>28 mm Hg). This is not unexpected but it demonstrates how stratification of risk on the basis of IOP is an important aspect of this and other referral refinement schemes, with the effect of such a stratification on inter-professional agreement reported for the first time in this study. The finding of improved PPPV for the identification of an abnormal IOP between OSI and consultant than between non-OSI and OSI can at least in part be explained by the standardisation of IOP measurement technique (Goldmann tonometry) between OSI and consultant in this scheme. The original JCG did not improve the PPPV for abnormal IOP assessments for both low and high risk referrals, although the updated JCG, where clearer guidance was given to referring optometrists on how IOP should be measured with both Goldmann and non-contact tonometry, did lead to an improvement in the PPPV in both groups.
PPPV between the OSI and consultant for identification of an abnormal optic disc demonstrated a statistically significant decline post-NICE (60.6%–42.7%). This is an interesting finding as dilated disc assessment with indirect ophthalmoscopy has been used since the introduction of this particular scheme. The lack of legal indemnity for optometrists not complying with the AOP's recommendation to the NICE guidelines may have led optometrists to adopt a more risk averse and medico-legally defensive approach to practice. The above finding suggests this may also be true for the OSI group with respect to optic disc assessment though not for IOP assessment. In addition, continued education and training of OSIs is important to maintain high clinical standards.
The good agreement between OSI and consultant for the sample of discharged patients implies that normal observations are being correctly identified; however, a larger sample is needed to make firm conclusions.
There are some limitations that warrant discussion. It has been assumed that the findings of the consultant ophthalmologists with specialist interest in glaucoma are the gold standard and therefore are correct. A similar assumption is made for the OSI in the review of low risk referrals from a non-OSI. Additionally, the analysis of non-OSI and OSI agreement for the pre-NICE period was based on only 16 patients which is a smaller sample size than the other 2-month periods analysed. This relatively small sample reflects the lower activity of the CHANGES scheme in this period compared with the post-NICE period. In addition, the sample used to calculate correlations for normal examination findings can only be an estimate as information was only gathered from patients who were referred to secondary care, and therefore not representative of the total patient population seen by the OSI and non-OSI.
Another potential limitation to the study but one that does reflect clinical practice is that the examination techniques employed by the non-OSI may differ from that of the OSI or consultant. Typically, a non-OSI will not dilate the pupil to perform an examination of the optic disc, whereas the OSI and consultant would always dilate the pupil except in the presence of an occludable anterior chamber angle. By way of a national survey of UK community optometrists, Myint et al reported that 25% of optometrists used direct ophthalmoscopy alone and a further 62% used a combination of direct ophthalmoscopy and slit-lamp binocular indirect ophthalmoscopy, whereas the OSI or consultant in the CHANGES scheme only used slit-lamp binocular indirect techniques.3 A difference in the method used for measuring IOP also exists, with the OSI and consultant solely using Goldmann applanation tonometry. Therefore, the agreement values involving the non-OSI in this report are not necessarily a comparison of two comparable examination techniques.
Summary
The accuracy of examination for detection of an abnormal IOP by the OSI has remained unchanged since 2006, though a decline was observed for the detection of an abnormal optic disc. For the non-OSI, there was a decline in accuracy in the detection of both abnormal IOP and optic disc assessment. The appropriate referral rate for an OSI decreased following the publication of the NICE glaucoma guidelines, though the appropriate discharge rate increased. A further report from the Health Innovation & Education Cluster Glaucoma Pathways project will address how these findings in conjunction with a detailed analysis of unnecessary referral temporal trends based on reason for referral will impact on glaucoma referral refinement scheme triaging criteria.
Acknowledgments
Assistance with searches for electronic patient record data was gratefully received from Tony Brenton of Medisoft. The authors thank the eight optometrists with a specialist interest in glaucoma (Sue Martynski, Caroline Hurst, Alison Hammond, Lesley Hodgson, Sally Bushby, Angela Wimpenny, John Kidd, Keziah Latham) and the community eye services department at Hinchingbrooke Hospital (Alison Hammond and Bridget Wilson).
References
Footnotes
-
Contributors All authors fully meet the criteria for authorship defined by the International Committee of Medical Journal Editors.
-
Funding This work is attributed to the North East, North Central London and Essex Health Innovation & Education Cluster Glaucoma Pathways project, for which a salary was provided for the research fellow. This research received no specific grant from any funding agency in the public, commercial or not-for-profit sectors. Prof Garway-Heath received a proportion of his funding from the Department of Health's National Institute for Health Research (NIHR) Biomedical Research Centre for Ophthalmology at Moorfields Eye Hospital NHS Foundation Trust and the UCL Institute of Ophthalmology. Prof Garway-Heath's chair at UCL is supported by funding from the International Glaucoma Association.
-
Competing interests None.
-
Ethics approval Hospital trust ethical approval was obtained.
-
Data sharing statement Further data can be made available from the corresponding author at g.ratnarajan@gmail.com.
-
Provenance and peer review Not commissioned; externally peer reviewed.
Linked Articles
- Editorials