Article Text
PDF
Abstract
Introduction There are 161 million people living with vision impairment, due to uncorrected refractive errors. A further 510 million people are living with near-vision impairment. There is a need for clearly defined indicators that capture the quality of refractive error service outputs and outcomes and provide insights to shape, change and stimulate action. This study aims to evaluate the quality of refractive error care (Q.REC) in Cambodia, Malaysia and Pakistan, by using unannounced standardised patients (USPs) to identify the proportion of prescribed and dispensed spectacles appropriate for people’s refractive error needs and pinpoint/detail opportunities for quality improvement.
Method and analysis A cross-sectional Q.REC study will be conducted in randomly selected optical services in Cambodia (180 services, 900 USP visits), the Klang Valley in Malaysia (66 services, 198 USP visits) and in Jhang, Sahiwal and Khanewal districts of Punjab region/state in Pakistan (64 services, 256 USP visits). USPs will receive baseline refractions by three skilled study optometrists/refractionists trained in the Q.REC protocol. USPs will then visit individual optical services, undergo a refraction, purchase spectacles or lenses (if recommended) and record observations about which elements of a refraction and dispensing were conducted. The study optometrist/refractionist will assess each pair of dispensed spectacles by examining the USP’s aided visual acuity and visual comfort at distance and/or near and compare the lens prescription to the averaged baseline refraction.
Ethics and dissemination This study has been approved by the University of New South Wales Human Research Ethics Committee (HC210102), the National Ethics Committee for Health Research in Cambodia (043 NECHR), National Medical Research Registry and the Medical Research and Ethics Committee (NMRR-21-689-59279) in Malaysia and the College of Ophthalmology & Allied Vision Sciences Ethical Review Board (COAVS 545/2021) in Pakistan. Written informed consent will be obtained from USPs. Service owners will have the opportunity to opt-out verbally or in writing. Results will be disseminated locally through workshops including the relevant local ministry of health personnel and stakeholders, published in peer-reviewed publications and presented at national and international conferences.
- quality in healthcare
- public health
- ophthalmology
- epidemiology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This multisite study evaluates the ‘real-world’ quality of refractive error care and identifies specific opportunities for quality improvement.
The ‘gold standard’ for evaluating clinical practice quality will be used, employing unannounced standardised patients (USPs) with a range of refractive error profiles.
As all USPs will be adults, the findings may not be applicable to the quality of children’s refractive error care.
As only an immediate assessment of comfort with spectacles is included, any subsequent positive or negative adaptation cannot be assessed.
Although USPs are trained to present their eye health history in a clear way, USPs may inadvertently mislead optical service staff and give an unclear version of their symptoms.
Introduction
Uncorrected refractive errors are the leading global cause of vision impairment, with 161 million people living with distance-vision impairment and an additional 510 million people living with near-vision impairment.1 The WHO World Report on Vision states that integrated people-centred eye care, and a commitment to universal health coverage, is the model of choice for quality and affordable eye care services and quality eye care services need to be provided according to population needs.2 The Lancet Global Commission on Global Eye Health argues that universal health coverage is not universal without affordable, high-quality, equitable eye care.3 In 2021, the United Nations General Assembly adopted a resolution committing the international community to provide eye health for the 1.1 billion people living with vision impairment by 2030.4 To be able to monitor progress towards universal health coverage and the quality of eye care services, a broader set of measurement indicators are required to monitor progress and drive change. Indicators should provide insights to shape change and stimulate action, track outcome progress and the quality of an intervention.3
Indicators have recently been established,5 to assess the ‘real-world’ quality of refractive care. These quality of refractive error care (Q.REC) criteria provide information on the proportion of prescribed spectacles that are clinically optimal for patients, by comparing a baseline prescription with dispensed spectacles. The Q.REC criteria were tested in Vietnam, in 93 optical stores and 480 individual visits, and demonstrated a strong association between the criteria for optimally prescribed spectacles and both vision and comfort.
This Q.REC indicator can also be used to assess core dimensions of quality, such as whether refractive error services are effective, equitable, safe and people centred, and have the potential to identify particular aspects of clinical practice that require improvement or further training.
A Q.REC study employs simulated patients—the gold standard for evaluating quality in clinical practice6—and can be easily administered in urban settings. Simulated patients, or unannounced standardised patients (USPs), are ‘actors’ who are trained to act covertly as patients in a standardised fashion, while observing clinical techniques and services provided. USPs have been used extensively in low-income and middle-income countries, often in evaluating family planning, pharmaceutical dispensing patterns and clinical prescribing patterns.7 Studies employing USPs have also previously been conducted to evaluate refractive error outcomes.5 8 9 If executed well, the distinct advantage of this method is that observation bias is minimised, as care providers are likely to modify their behaviours if they feel that they are being observed or examined.
A pilot Q.REC study in Vietnam in 2018 found that out of 417 pairs of spectacles purchased from 93 optical outlets, only 44% of spectacles resulted in both good vision and comfort for patients.5 This highlights a significant gap in current models of care and represents an opportunity for improving quality outcomes within the existing infrastructure.
It is anticipated that a Q.REC study will identify the proportion of people who are prescribed and dispensed spectacles appropriate for their refractive error needs and identify specific opportunities for quality improvement which can then be translated into policy changes or quality improvement initiatives. Q.REC studies can also be used to monitor ongoing delivery of quality refractive error care services, within the context of integrated people-centred eye care.
The overall objective of this study is to evaluate the Q.REC in Cambodia, Malaysia and Pakistan. The primary outcome of interest is the proportion of spectacles dispensed from optical service providers that meet ‘optimal’ quality standards, as defined by Lee et al.5 Secondary outcomes of interest are the association between ‘optimal’ quality spectacles and refraction/dispensing techniques, subjective visual acuity and USP’s comfort while wearing dispensed spectacles.
Methods and analysis
Study setting
In each location we have drawn a representative sample of eligible optical services using simple random sampling. The sampling frame is comprised of all optical services within provinces with more than five optical services in Cambodia, all optical services within the Klang Valley in Malaysia and all optical services within the Jhang, Sahiwal and Khanewal districts of Punjab in Pakistan. It is anticipated that enrolment will commence in October 2021 and will be completed by June 2022.
Study population
The inclusion criteria for USPs are adults aged 18 years or above, fluent native speaker of the primary language of the district, good ocular health and a refractive error profile of interest. This includes adults who need spectacles, as well as adults who have emmetropic eyes and do not need spectacles. We will exclude USPs who have had prior refractive eye surgery, eye surgery within the prior 3 months, USPs with manifest or intermittent strabismus, USPs with amblyopia and USPs who have any ocular or health conditions that can cause variable spectacle prescription. Each USP will order glasses from each service and may visit multiple services.
Eligible optical services are those that provide refractive error services in Cambodia, Malaysia (Klang Valley) and Pakistan (Jhang, Sahiwal and Khanewal districts of the Punjab province). Optical services that are part of a franchise or chain will be considered a different service. Optical services will be excluded from the study if the optical service is personally known to the USP or if it is identified that the optical service has staff working across multiple services that have already been selected. The participant information statement and withdrawal form will be hand delivered or sent via registered post, in the local language, at least 1 month in advance advising that the services might be visited for research purposes. During this time, the optical service owner/manager will be asked to read the form, consider whether they would like to opt-out of the study and identify whether study feedback is desired. If the randomly selected services do not return the form for withdrawal of participation by the due date and the first USP attending says they are unlikely to be detected, the services are considered successfully enrolled. As the public (potential optical service clients/patients) have a right to understand the quality of the services that they might expect to receive in each location, an opt-out approach will be used to ensure that there is a high participation rate. Also, the research is likely to be compromised if optical stores are aware that they are providing optical services to a USP due to the Hawthorne effect (where clinicians modify their behaviour in response to being observed). The quality of each individual store will not be published in any way so the privacy of each store will be maintained.
Sampling strategy
Lists of all eligible services were compiled by consulting with relevant ministries of health, optical councils, optometry associations and cross-referencing with Google Maps. Services will be selected by computer-generated simple random sampling by study coordinators after the due date to withdraw from participation.
Sample size
The anticipated proportion of spectacles of optimal quality (p) was chosen based on previous studies and local knowledge (see table 1).5 9 Malaysia has a more developed and regulated optometry industry than Cambodia or Pakistan, so it was anticipated that there the proportion of spectacles that are of optimal quality was higher. The margin of error for the 95% CI was chosen to provide sufficient precision with the available resources, as each study was funded separately. For Malaysia and Pakistan, the desired margin of error was set at 7%. In Cambodia, margin of error was set at 4% as the study budget provided the opportunity to maximise the sample size, so as to reduce the margin of error and make additional learnings on the precision of the intraclass coefficient. The intraclass correlation was estimated to be 0.1, and the design effect was estimated to be 1.2. Based on the parameters presented in table 1, a total of 1354 spectacles will be purchased.
Parameters for sample size estimates
It is anticipated that up to 40 USPs will be employedand each USP will be able to visit three to four services per day.
Development of USPs
Recruitment of USPs
To recruit USPs, study flyers have been advertised in ophthalmology outpatient clinics and at adult education institutions in local languages. Participating USPs will be reimbursed for participation, plus any travel, accommodation, meal or spectacle purchase expenses.
Training
All USPs will be briefed/trained in person on the tasks involved by a study coordinator and study optometrist, and will be allowed sufficient time to practice acting as a patient. Training packages have been developed to provide USPs with a ‘script’ that they can use during optical service visits. Post-training observations by a study optometrist will be conducted to identify whether the USPs can accurately identify elements of refraction and dispensing techniques. If USPs are unable to accurately identify elements of refraction and dispensing techniques, further training will be provided.
Baseline refraction
Baseline refractions of each USP will be conducted by three skilled study optometrists/refractionists who will be recruited within each country to conduct individual clinical refractions, in order to establish the USP’s average baseline prescription and their best-corrected visual acuity. All USPs will undergo three baseline refractions by the study optometrists/refractionists. For USPs with significant refractive error where a modified prescription might be required for adaptation, the study optometrists/refractionists will agree on level of modification that is to be considered for the baseline refraction, otherwise full prescription will be considered for baseline. If the refraction components (spherical and cylindrical powers) are not within 0.75 D of each other, a fourth refraction is required. The most senior optometrist will then decide which three refraction results will be used for the averaged baseline refraction for that USP.
Procedures
USP optical service visit
To assess the quality of the spectacles prescribed and dispensed, USPs will visit individual optical services, undergo a refraction, purchase spectacles (if recommended) and record observations about which elements of a refraction and dispensing were conducted (online supplemental file 1). Although a schedule will be developed for each USP to attend selected optical services, no prior appointment will be made unless it is standard practice for the optical service. Once the eye examination is complete, the USP will place an order for new spectacles/lenses. After the visit, the USP will find a suitable location to complete the optical services visit checklist electronically. The USP will return to each service to collect the purchased spectacles/lenses and obtain a copy of the spectacle prescription.
Supplemental material
Spectacle quality
Each pair of spectacles dispensed will be assessed by the study optometrist/refractionist to determine whether the prescribed spectacles pass or fail the criteria for optimally prescribed spectacles (table 2) and assess the visual acuity and vision comfort at distance and near.
Criteria for optimally prescribed spectacles
Evidence-based spectacle quality criteria were developed from published mean subjective intolerance to spectacles for lens power and induced prism10 11 and published dispensing standards for tolerance to cylindrical axis12 (table 2). Achieving all criteria in both eyes with a pair of spectacles is defined as ‘optimally prescribed spectacles’.
Analysis
Spectacle characteristics
The distribution of spectacle characteristics will be compared between and across regions using Pearson’s χ2 test. Spectacle characteristic will include lens type (single vision near, single vision distance) and refractive error type (binary variable for each of presbyopia, astigmatism, myopia and hypermetropia).
Primary outcome
The unit of analysis will be one pair of spectacles. The proportion of spectacles that are ‘optimal’ within each country and district will be estimated using sampling weights, and logit-transformed 95% CIs will be estimated using robust standard errors to account for intraservice correlation.
Secondary outcomes
Service characteristics (refraction techniques and refraction equipment used), USP comfort and USP-corrected visual acuity while wearing dispensed spectacles (dichotomised as achieving best-corrected binocular visual acuity vs not achieving (worse than 1.5 lines on a logMAR chart)) will be compared between spectacles of optimal and suboptimal quality via logistic regression, adjusting for sampling weights and intraservice correlation.
Analyses will be conducted using Stata/BE V.17.0 (StataCorp LLC, College Station, Texas) or equivalent.
Database management
Study data will be collected, managed and securely stored using Research Electronic Data Capture hosted at The Fred Hollows Foundation.13 14 Study data will be entered into the database by the USPs and study optometrists/refractionists. Data quality will be assured by conducting daily queries to identify and resolve discrepancies and by using data quality rules.
Patient and public involvement
There was no patient or public involvement in the design of this protocol, although all coauthors have lived experience with refractive errors.
All participating USPs and service owners will have the opportunity to receive a summary of the results in their preferred language.
Ethics and dissemination
As specified above, enrolment of services in this study will be on an opt-out basis, with written information on the nature of the study being provided to the service owners. Services owners will be offered the opportunity to decline participation, verbally or in writing. Written informed consent will also be obtained from the USPs after the nature of the study and potential harms have been explained. Ethics approvals have been obtained from the following ethics committees:
Cambodia: National Ethics Committee for Health Research, Cambodia (043 NECHR).
Malaysia: National Medical Research Registry and the Medical Research and Ethics Committee (NMRR-21-689-59279).
Pakistan: College of Ophthalmology & Allied Vision Sciences Ethical Review Board (COAVS 545/2021).
Australia: University of New South Wales Human Research Ethics Committee (HC210102).
Results will be disseminated to local government partners through workshops to discuss potential study and policy implications. In addition, the results will be published in peer-reviewed publications and presented at national and international conferences.
Discussion
The Q.REC study aims to provide a much-needed methodology for assessing the Q.REC, as experienced by patients. This study will identify the proportion of spectacles obtained from optical services that provide quality vision and comfort outcomes and the associations between spectacle quality and refraction and dispensing techniques. The results of this study will provide valuable information on the Q.REC and identify specific opportunities for quality improvement in Cambodia, Malaysia and Pakistan. This information will provide much-needed empirical evidence to aid policy-makers, education institutions and service providers on how to ensure that refractive error services provide quality care. The findings can be translated into policy changes or quality improvement initiatives and additional Q.REC studies can be used to monitor progress towards delivering quality refractive error care.
The Lancet Global Health Commission on Global Eye Health stated that concerted efforts are needed to improve quality eye health outcomes and reliable survey and service data should be available and used by implementers and policy-makers to monitor progress.3 A Q.REC study provides clearly defined indicators that capture refractive error service outputs and outcomes, along with ‘insights to shape change and stimulate action’.3 In addition, a Q.REC study includes broader dimensions of refractive error quality. Using visual acuity alone as the indicator of refractive error quality does not take into account components of spectacle quality that can exacerbate binocular vision disorders or contribute to visual discomfort. Nor does visual acuity alone take into account timeliness, visual comfort, effectiveness, safety (eg, spectacle prescriptions that result in unnecessarily reduced vision) or any assessment of the extent to which services are people centred. These additional aspects of quality care are a core component of advancing universal health coverage through eye care.
There are several limitations to this study that should be acknowledged. First, as all USPs will be adults, the findings may not be applicable to the quality of children’s refractive error care. Second, USPs may inadvertently mislead optical service staff and give an unclear version of their symptoms. To reduce this potential bias training will be provided to ensure USPs are confident at presenting symptoms and history in a clear manner. Third, it could be argued that by only including an immediate assessment of comfort, the possibility of any positive or negative adaptation that might occur with extended wear is ignored. However, this methodology is not designed to assess spectacle compliance, which is likely to be heavily impacted by adaptation that occurs during extended wear. Although prior studies have indicated that spectacle wear compliance rates can be as low as 40%,15 which infers that as many as 60% of dispensed spectacles remain unworn, this study may provide evidence on the proportion of spectacles that are likely to be unsuitable for population needs. These insights can be used to stimulate action and provide refractive error services according to population needs—resulting in higher rates of spectacle wear.
Currently, the Q.REC indicators are focused on whether they are people centred, effective, equitable and safe. Additional questions and components could be added to assess other dimensions of quality, such as whether they are timely, integrated and efficient. Future trials could also include more aspects of dispensing techniques, such as frame fit and comfort.
This study will produce a set of validated tools for assessing the quality of spectacles obtained from optical services, resulting in solution-focused contextually relevant research that will encourage high quality and universal eye health for all.
Ethics statements
Patient consent for publication
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors AMB and LL led the design of the study and developed the protocol. AMB applied for ethical approvals and drafted the manuscript. MM contributed to design of statistical analysis methods, sample size calculations and drafting the protocol. BV, YPH and SMH provided advice on key study issues and feedback on the protocol. All authors read, revised and approved the final manuscript.
Funding This work was funded by The Fred Hollows Foundation with support from the Australia Government through the Australian NGO Cooperation Program.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.