Article Text

PDF

Cross-sectional comparison of point-of-care with laboratory HbA1c in detecting diabetes in real-world remote Aboriginal settings
  1. Julia V Marley1,2,
  2. May S Oh2,
  3. Nyssa Hadgraft2,
  4. Sally Singleton2,
  5. Kim Isaacs2,
  6. David Atkinson1,2
  1. 1The Rural Clinical School of Western Australia, The University of Western Australia, Broome, Western Australia, Australia
  2. 2Kimberley Aboriginal Medical Services Council, Broome, Western Australia, Australia
  1. Correspondence to Dr Julia Marley; Julia.Marley{at}rcswa.edu.au

Abstract

Objectives To determine if point-of-care (POC) glycated haemoglobin (HbA1c) is sufficiently accurate in real-world remote settings to predict or exclude the diagnosis of diabetes based on laboratory HbA1c measurements.

Design Cross-sectional study comparing POC capillary HbA1c results with corresponding venous HbA1c levels measured in a reference laboratory.

Participants Aboriginal patients ≥15 years old who were due for diabetes screening at the participating clinics were invited to participate. Two hundred and fifty-five Aboriginal participants were enrolled and 241 were included in the analysis.

Setting 6 primary healthcare sites in the remote Kimberley region of Western Australia from September 2011 to November 2013.

Main outcome measures Concordance and mean differences between POC capillary blood HbA1c measurement and laboratory measurement of venous blood HbA1c level; POC capillary blood HbA1c equivalence value for screening for diabetes or a high risk of developing diabetes; sensitivity, specificity and positive-predictive value for diagnosing and screening for diabetes; barriers to conducting POC testing.

Results Concordance between POC and laboratory results was good (ρ=0.88, p<0.001). The mean difference was −0.15% (95% limits of agreement, −0.67% to 0.36%). POC HbA1c measurements ≥6.5%, 48 mmol/mol had a specificity of 98.2% and sensitivity of 73.7% for laboratory measurements ≥6.5%. The POC equivalence value for screening for diabetes or a high risk of developing diabetes was ≥5.7%, 39 mmol/mol (sensitivity, 91%; specificity, 76.7% for laboratory measurements ≥6.0%, 42 mmol/mol). Staff trained by other clinic staff ‘on the job’ performed as well as people with formal accredited training. Staff reported difficulty in maintaining formal accreditation.

Conclusions POC HbA1c testing is sufficiently accurate to be a useful component in screening for, and diagnosing, diabetes in remote communities. Limited local training is adequate to produce results comparable to laboratory results and accreditation processes need to reflect this.

  • Aboriginal
  • point-of-care testing
  • diabetes screening
  • HbA1c

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

Statistics from Altmetric.com

Strengths and limitations of this study

  • This was a pragmatic study conducted in ‘real-world’ remote clinics where staff were trained realistically in-house to use the point-of-care glycated haemoglobin (HbA1c) analyser.

  • This study provides insight into how simplified screening for and diagnosing diabetes using HbA1c tests can be implemented in remote primary healthcare.

  • Not all participants with abnormal laboratory HbA1c measurements were able to be located for follow-up repeat laboratory HbA1c tests.

  • We used a ‘convenience sample’, which may not be representative of the Kimberley adult Aboriginal population.

Introduction

Screening for diabetes in remote Aboriginal Australian settings is challenging, in part due to the difficulty of obtaining fasting blood glucose levels and then completing an oral glucose tolerance test in patients where the glucose level cannot exclude or diagnose diabetes.1 In 2009 and 2011, the American Diabetes Association (ADA) and WHO, respectively, updated their criteria for diagnosing diabetes to include glycated haemoglobin (HbA1c) testing.2 ,3 The potential advantages of using HbA1c include no need for a fasting sample.3 The correlation between HbA1c levels and presence of retinopathy was used by the ADA and WHO to determine the HbA1c threshold of ≥6.5% (48 mmol/mol) to diagnose diabetes.3 ,4 ADA also uses HbA1c values 5.7–6.4% (39–46 mmol/mol) to identify people at future risk of diabetes with 6.0–6.4% (42–46 mmol/mol) classified as high risk.

In Australia, the National Evidence Based Guideline for Case Detection and Diagnosis of Type 2 Diabetes was last updated in 2009 and is based on the 2006 WHO criteria for diagnosing diabetes.5 ,6 It currently does not yet recommend using HbA1c testing for diagnosis. However, the Australian Medicare Benefits Schedule (MBS) has recently changed to include funding for laboratory HbA1c testing for diagnosing diabetes from 1 November 2014.7

Laboratory HbA1c tests for patients in the Kimberley region of north Western Australia are analysed in Perth, the state capital, over 1600 km away. This can lead to significant delays in receiving test results for patients located in remote areas of the Kimberley. Once Australian guidelines incorporate the use of HbA1c testing for diagnosing diabetes, the availability of immediate HbA1c results is likely to further improve diagnosis in remote areas and timeliness of initiating management, including lifestyle advice and pharmacotherapy.

In a number of studies, point-of-care (POC) testing of HbA1c has been shown to be closely correlated with laboratory-based results, and the process is well accepted in Aboriginal Medical Services across Australia.8–11 These studies, in supported research environments with attention to quality control, have not however been repeated in real-world remote settings where maintaining consistency of quality and accreditation may be more difficult. POC HbA1c testing in Australia is currently used in the management of diabetes, and accreditation with a quality assurance programme, such as the Quality Assurance for Aboriginal and Torres Strait Islander Medical Services (QAAMS) programme, is required for clinics to claim MBS rebates for these tests.12 Advice submitted to the Medical Services Advisory Committee also recommends that POC HbA1c testing be funded for diabetes screening if it is performed on an analyser accredited in a recognised external quality assurance programme such as QAAMS;13 however, this has not as yet been accepted.

An earlier Kimberley study of POC testing, comparing capillary glucose with venous laboratory glucose, demonstrated that real-world remote primary healthcare delivered accurate POC testing of glucose in the diagnosis of diabetes without formal training.14 Following this study, the diagnostic pathway in the Kimberley was modified, and the Kimberley Chronic Disease Therapeutic Protocol for type 2 diabetes was updated to reflect these changes.15 ,16 The rationale for POC HbA1c screening for diabetes is at least as persuasive given the difficulty and delay in accessing laboratory services in remote settings and the benefits of immediate results without the need for retesting in a mobile population.

We aimed to determine if POC capillary blood HbA1c measurement was sufficiently accurate, in a real-world setting using the existing processes for HbA1c assessment, to reliably predict or exclude the diagnosis of diabetes based on laboratory venous blood HbA1c measurements in remote Aboriginal primary healthcare practice.

Methods

Data were collected by local healthcare providers from September 2011 to November 2013 at six sites across the Kimberley. All Aboriginal and Torres Strait Islander people ≥15 years old in the Kimberley are considered to be at high risk of developing diabetes and local protocols recommend that they are screened annually.15 Aboriginal and/or Torres Strait Islander people who were due for diabetes screening at the participating clinics were invited to participate. The study was carried out on an opportunistic basis when sufficient clinic resources were available to enrol participants.

POC HbA1c measurement and venepuncture

Blood was expected to be collected for capillary and venous HbA1c on the same visit; however, this did not always occur. In some remote settings in the Kimberley, samples may not leave the clinic for up to a week and venous samples are often collected just prior to transport. Results up to 7 days apart were therefore included. Data for comparison of POC and laboratory HbA1c measurements were excluded if the participant's haemoglobin was <90 g/L.

Capillary blood HbA1c was measured by a primary healthcare provider via a finger-prick blood sample and analysed on a DCA 2000+ Analyzer (Siemens/Bayer, Germany), the only type of POC machine available in Kimberley clinics during the study and used routinely for assessing diabetic control. The study aimed to look at ‘real world’ practice and hence did not attempt to change the way staff learnt how to do the testing or maintain and calibrate the machines. We collected data on clinic processes and calibration after the study stopped recruiting participants by contacting each clinic involved.

Venous whole blood samples were collected in containers with EDTA. Normal procedures at each clinic were used for storage (whole blood was stored at 4°C) and whole blood samples were transported to one of the three laboratories in the Kimberley region before being transported to the main laboratory in Perth (PathWest). The distance to the Kimberley laboratories from the study sites ranged from 2 km by road to 650 km by air, and from the Kimberley laboratories to Perth from 1677 to 2224 km by air. The time for the specimens to reach the Perth laboratory varied from 1 to 8 days.

Venous plasma HbA1c levels were measured as part of routine work by PathWest (which analyses most Kimberley pathology). PathWest used automated immunoassay with anticoagulated whole blood specimens haemolysed automatically on the Cobas Integra 800 (Roche Diagnostics, Switzerland) using HbA1c haemolysis reagent in the predilution cuvette. Total haemoglobin was measured colourimetrically while HbA1c was determined immunoturbidimetrically. The ratio of the concentrations of total haemoglobin and HbA1c in the specimen yielded the final proportionate HbA1c measurement.

Readers of the POC and laboratory HbA1c test results were blinded to the other HbA1c test results. POC HbA1c test results were recorded by clinic staff on the participant's data collection form. Laboratory HbA1c test results were sent by PathWest to the requesting doctor after the POC test was conducted. Participants with laboratory HbA1c ≥5.7%, 39 mmol/mol were expected to be followed up with another laboratory HbA1c test to confirm prediabetes or diabetes.

Sample size calculation

On the basis of pilot screening data from the Kimberley, we estimated that prevalence of undiagnosed diabetes is 15% (according to Martin et al8 up to 20%) and if HbA1c has a sensitivity of 60%,17–20 then 9% would be diagnosed by laboratory HbA1c. The size of the sample needed depends on the accuracy of sensitivity and specificity required.21 Assuming a minimum sensitivity of 90% and specificity of 80% for screening for diabetes (and vice versa for diagnosing diabetes), using calculations by Buderer22 we needed a sample size of 138 and 270 to detect a two-tailed 95% CI with a maximum width of ±5% for sensitivity (ie, 85–95%) and specificity (ie, 75–85%), respectively.

Statistical analysis

Differences in baseline characteristics were compared using χ2 tests for categorical data and Mann-Whitney tests for continuous non-parametric data.

Concordance between POC and laboratory blood HbA1c measurements was assessed by the technique of Lin,23 which measures the agreement between two methods. Mean difference (bias) and limits of agreement were determined using the techniques of Bland and Altman.24 Linear regression analysis was used to determine whether participant or processing factors affected the relationship between POC and laboratory HbA1c levels.

The POC HbA1c diagnostic cut point was based on the laboratory HbA1c diagnostic cut point recommended by the Australian Diabetes Society expert committee (≥6.5%, 48 mmol/mol).25 The Kimberley POC capillary equivalence value for screening for people with diabetes or at high risk of developing diabetes was determined from receiver operating characteristics (ROC) curves, using a laboratory value of 6% (42 mmol/mol), based on International Expert Committee and ADA recommendations for people at high risk of developing diabetes.3 ,4 Sensitivity, specificity, predictive values and percentage correctly classified for (1) diagnosing diabetes and (2) screening for diabetes and a high risk of developing diabetes were determined based on these cut points.

All analyses were performed using Stata, V.13 (StataCorp, Texas, USA). Point estimates were presented with 95% CIs; p<0.05 was considered statistically significant.

Results

Two hundred and fifty-five participants were enrolled, and 241 included in the analysis. The reasons for exclusion were missing tests in 12 participants, and more than 7 days elapsing between POC measurement and venepuncture in two participants. There was no difference in age (median (range): 36 (17–79) vs 38.5 (17–71); p=0.397) or sex (female (%): 143 (59%) vs 9 (64%); p=0.714) between those included and excluded from the analysis.

Unadjusted POC capillary blood HbA1c measurements and laboratory-measured venous blood HbA1c levels showed good concordance (figure 1A). The limits of agreement between POC and laboratory measurements (mean difference: −0.15%; 95% limits of agreement: −0.67% to 0.36%) are presented in figure 1B. The cross tabulation of the classification of participants based on the POC and HbA1c measurements is shown in table 1.

Table 1

Cross tabulation of the classification* of participants (n=253) based on POC and laboratory HbA1c measurements

Figure 1

Concordance analysis of point-of-care (POC) and laboratory glycated haemoglobin (HbA1c) results. (A) All unadjusted readings (plasma and whole blood levels; n=241). Dashed line indicates line of perfect concordance. (B) Difference plot (Bland-Altman analysis)24 for all POC versus laboratory HbA1c. Dotted line indicates 95% limits of agreement.

In the regression analysis, the relationship between POC and laboratory HbA1c levels was not independently influenced by any of the following: the time between POC HbA1c measurement and venepuncture; the time delay in collecting and processing the laboratory specimen; whether the health worker had performed more than 10 POC tests; health worker type (Aboriginal health worker, nurse, general practitioner); site; or if the POC operator was previously or currently accredited with QAAMS.

Accuracy of POC HbA1c in predicting diabetes

ROC curves for diagnosing diabetes (laboratory HbA1c level ≥6.5%, 48 mmol/mol) are shown in figure 2A. Sensitivity, specificity, predictive values and percentage correctly classified for POC and laboratory measurements ≥6.5%, 48 mmol/mol are listed in table 2. As required for diagnostic tests, POC HbA1c testing had a high specificity (98.2%; 95% CI 95.1% to 99.4%) for predicting diabetes.

Table 2

Sensitivity, specificity, predictive values and classification by point-of-care (POC) glycated haemoglobin (HbA1c) testing for diagnosing diabetes and screening for participants (n=241) with diabetes or a high risk of developing diabetes

Figure 2

ROC curve for diagnosing and screening for diabetes or a high risk of developing diabetes. ROC curve for (A) diagnosing diabetes (laboratory-measured HbA1c level, ≥6.5%, 48 mmol/mol) and (B) screening for diabetes or a high risk of developing diabetes (laboratory-measured HbA1c level ≥6.0%, 42 mmol/mol; HbA1c, glycated haemoglobin; POC, point-of-care; ROC, receiver operating characteristics).

For the dichotomous classification of participants as either not having (HbA1c <6.5%, <48 mmol/mol) or having diabetes (HbA1c ≥6.5%, ≥48 mmol/mol), based on laboratory measurements, nine of the 241 (3.7%) participants were incorrectly classified with POC HbA1c. The four participants who were incorrectly classified as having diabetes with POC HbA1c (6.6–6.8%, 49–51 mmol/mol) had laboratory HbA1c measurements in the prediabetes range (table 1), with two close to the cut point (6.3–6.4%, 45–46 mmol/mol). The other five participants had laboratory measurements just above the diagnostic cut point (6.5–6.7%, 48–50 mmol/mol) and POC measurements in the prediabetes range (table 1), with four close to the cut point (6.3–6.4%, 45–46 mmol/mol).

Of the 14 participants who had both POC and laboratory measurements ≥6.5%, 48 mmol/mol, 8 were followed up with a repeat laboratory HbA1c test. The second laboratory result confirmed a diagnosis of diabetes in 7 (88%) participants, with the other participant having a measurement close to the cut point (6.3%, 45 mmol/mol).

Accuracy of POC HbA1c in screening for diabetes or a high risk of developing diabetes

As a screening test, the most clinically appropriate POC threshold based on ROC curves was 5.7%, 39 mmol/mol (figure 2B). Sensitivity, specificity, predictive values and percentage correctly classified for POC measurements ≥5.7%, 39 mmol/mol and laboratory measurements ≥6.0%, 42 mmol/mol are listed in table 2. As required for screening tests, POC HbA1c testing had a high sensitivity (91.0%) for detecting people with diabetes or a high risk of developing diabetes.

Using this cut point of 5.7%, 39 mmol/mol, 132 (55%) participants were classified as normal with POC testing. No participant with a diagnostic diabetes laboratory HbA1c measurement was missed, and only 7 (2.9%) participants would have had a high risk of developing diabetes (HbA1c ≥6.0%) missed if POC HbA1c was the only test undertaken.

Accreditation and quality assurance of POC HbA1c testing

Only two staff members were accredited with QAAMS to perform POC HbA1c tests for managing diabetes12 when participating in the study. Three other staff involved in the study had previously received QAAMS accreditation (2004) but were no longer accredited. Many staff were trained to use the analyser by other clinic staff who, in turn, had experience ranging from minimal through to QAAMS accreditation. If no one in the clinic was familiar with the machine, staff taught themselves by reading the instruction poster. Only one site had QAAMS accreditation during the study period and this site calibrated the machine as required to maintain accreditation (two monthly quality checks: internal quality control and external quality assurance testing).12 An analyser was moved from one remote location to another and was calibrated by staff at the site with QAAMS accreditation prior to being used in the study. Staff from the other four sites, where 140 (59%) samples were tested, reported that they did not calibrate their analyser during the study period.

Barriers to POC testing

Barriers to POC HbA1c testing included several sites needing new analysers, and in 2011 there was a global shortage of cartridges. Staff reported that they found maintaining accreditation through QAAMS difficult and the quality management framework required for the clinic to maintain QAAMS accreditation12 was seen as onerous. In the Kimberley, cartridges are usually bought by clinics through the QAAMS programme and staff reported that if their clinic was not accredited, they were unable to order cartridges and claim MBS rebates. These issues meant that at the start of the study only one clinic was doing routine POC HbA1c testing for diabetes monitoring.

Discussion

Our study demonstrated good concordance between laboratory and real-world remote setting POC HbA1c results. Measurements using the DCA 2000+ Analyzer were similar to, but slightly lower on average, than laboratory venous samples measured routinely by PathWest. A POC cut point of 6.5%, 48 mmol/mol gave a high specificity (98.2%) for diagnosing diabetes based on the same cut point for laboratory measurements.

We also found that a POC HbA1c screening cut point of 5.7%, 39 mmol/mol had a high sensitivity for detecting an HbA1c level ≥6.0%; 42 mmol/mol (91.0%) and detected all participants with diagnostic diabetes laboratory HbA1c measurements and most participants in the prediabetes range (ie, those at high risk of developing diabetes in 5 years).26–28 As all Aboriginal and Torres Strait Islander people ≥15 years old in the Kimberley are considered to be at high risk of developing diabetes, misclassification of the seven participants with prediabetes would have limited impact on clinical management. The main interventions for prediabetes are encouraging healthy lifestyles and these should be offered to all patients. In addition, local recommendations for annual screening15 increase the prospects of progression to diabetes being diagnosed.

If the study population was reasonably typical of Kimberley Aboriginal and Torres Strait Islander residents undergoing screening for diabetes, a POC HbA1c cut point of <5.7%, 39 mmol/mol to exclude diabetes will reduce the need for venepuncture and a follow-up visit to the clinic to discuss test results by 55%. This is more than twice the reduction found in the Kimberley POC glucose study.14

Various expert committees suggest that for asymptomatic diagnostic results a confirmatory test should be performed when feasible to confirm diabetes.4 ,25 ,29 In contrast, the Australian Medical Services Advisory Committee does not consider a confirmatory test to be necessary,30 and there is only one MBS rebate for laboratory HbA1c for screening in any 12-month period.7

If MBS rebates were available for screening using POC HbA1c analysers in Australia, a POC measurement ≥5.7%, 39 mmol/mol could be followed by a venous laboratory HbA1c test on the same day. If the laboratory result was in the diagnostic range (≥6.5%, 48 mmol/mol), this would confirm a diagnosis of diabetes in Australia. Given the lower than expected sensitivity (73.7%) for detecting diabetes, it is likely that some patients with POC measurements just below the diagnostic cut point will have a laboratory result in the diagnostic range. Therefore, in high-risk remote populations in countries where a confirmatory test is required if only the venous laboratory result (taken on the same day as the POC measurement) was ≥6.5%, a second laboratory test would need to be performed to confirm the result. Expediting the process and reducing the likelihood of loss to follow-up could result in more timely initiation of management in primary healthcare.

With the dichotomous classification of participants, they were nearly all correctly classified. Of the participants who had initial diagnostic POC and laboratory measurements followed up, a second HbA1c laboratory measurement confirmed the diagnosis of diabetes in all except one, who had a repeat laboratory measurement close to the cut point. The clinical treatment for these patients is similar (healthy lifestyle advice unless HbA1c ≥7%, 53 mmol/mol), with the main differences being frequency of follow-up and screening for microvascular disease.

A limitation of this study is that not all participants with laboratory HbA1c ≥5.7%, 39 mmol/mol were followed up with repeat laboratory HbA1c tests. A further limitation is that participation was based on inviting patients visiting participating clinics to enrol in the study, that is a ‘convenience sample’. On the basis of population surveys, we predicted that the number of people with undiagnosed diabetes across a range of remote Kimberley towns and communities would be 15%.8 We found that only 6% of the study population had previously undiagnosed diabetes. This lower than expected prevalence could be due to the study population not being representative of the Kimberley Aboriginal population or that there is less undiagnosed diabetes in the Kimberley than expected.

Unlike other POC HbA1c studies,8–11 our study was conducted in ‘real-world’ remote clinics where staff were mainly trained in-house, and no extra effort was made to standardise health-provider education, or HbA1c analyser maintenance or calibration. Results demonstrated that despite limited machine calibration and minimal training, POC results were good. The perceived difficulty of maintaining accreditation of staff and clinics to obtain MBS rebates for management of diabetes using POC analysers is a barrier to greater use of HbA1c for management of diabetes, and this will also be a barrier to POC screening. Although there are a variety of training methods provided by QAAMS, including online training, staff still need to be assessed by someone with current QAAMS accreditation in order to be accredited.12 This can be difficult in remote areas without easy access to trainers and where workforce turnover is high. The expectations of the quality management framework requirements were also seen to be onerous. While we advocate for, and do encourage, regular machine calibration, it appears that there would be minimal risks and considerable benefits if the quality management requirements and accreditation processes were simplified.

Implications

POC HbA1c analysers can be used in real-world remote Aboriginal primary healthcare settings for opportunistic screening for diabetes. In remote areas, using POC HbA1c measurement would expedite the diagnosis of diabetes in high-risk, remote-dwelling populations and provide opportunities for immediate education and initiation of management.

Acknowledgments

The authors thank the staff and patients of Broome Regional Aboriginal Medical Service, Derby Aboriginal Health Service, Yuri Yungi Medical Service and the Balgo, Bidyadanga and Warmun healthcare clinics for their participation in this study. They also thank Sharon Evans for expert statistical advice. They acknowledge that this project was supported by the Kimberley Aboriginal Health Planning Forum Research Subcommittee.

References

View Abstract

Footnotes

  • Contributors JVM and DA conceived and designed the study. MSO (September 2011 to August 2012), KI (August 2012 to July 2013) and SS (August 2013 to January 2014) managed the project as part of their GP registrar training. NH assisted in collecting and entering data into the databases. JVM conducted the statistical analysis and drafted the manuscript. All authors contributed to the interpretation of the findings and provided a critical review of the manuscript. All authors read and approved the final manuscript.

  • Funding This project was funded in part through a Rural Clinical School of Western Australia Small Project Grant.

  • Competing interests None.

  • Patient consent Obtained.

  • Ethics approval Human Research Ethics Committee of The University of Western Australia, the Western Australian Aboriginal Health Ethics Committee and WACHS Research Ethics Committee.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement No additional data are available.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.