Objectives In its guidelines on the use of portable monitors to diagnose obstructive sleep apnoea, the American Academy of Sleep Medicine endorses home polygraphy with type III devices recording at a minimum airflow the respiratory effort and pulse oximetry, but advises against simple pulse oximetry. However, oximetry is widely available and simple to use in the home. This study was designed to compare the ability of the oxygen desaturation index (ODI) based on oximetry alone with a stand-alone pulse oximeter (SPO) and from the oximetry channel of the ApneaLink Plus (ALP), with the respiratory disturbance index (RDI) based on four channels from the ALP to predict the apnoea–hypopnoea index (AHI) from laboratory polysomnography.
Design Cross-sectional diagnostic accuracy study.
Setting Sleep medicine practice of a multispecialty clinic.
Participants Patients referred for laboratory polysomnography with suspected sleep apnoea. We enrolled 135 participants with 123 attempting the home sleep testing and 73 having at least 4 hours of satisfactory data from SPO and ALP.
Interventions Participants had home testing performed simultaneously with both a SPO and an ALP. The 2 oximeter probes were worn on different fingers of the same hand. The ODI for the SPO was calculated using Profox software (ODISOX). For the ALP, RDI and ODI were calculated using both technician scoring (RDIMAN and ODIMAN) and the ALP computer scoring (RDIRAW and ODIRAW).
Results The receiver–operator characteristic areas under the curve for AHI ≥5 were RDIMAN 0.88 (95% confidence limits 0.81–0.96), RDIRAW 0.86 (0.76–0.94), ODIMAN 0.86 (0.77–0.95), ODIRAW 0.84 (0.75–0.93) and ODISOX 0.83 (0.73–0.93).
Conclusions We conclude that the RDI and the ODI, measured at home on the same night, give similar predictions of the laboratory AHI, measured on a different night. The differences between the two methods are small compared with the reported night-to-night variation of the AHI.
- SLEEP MEDICINE
- sleep apnea
- home testing
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
This is the first study that directly compares the diagnostic accuracy of a type IV with a type III home testing system to predict the apnoea–hypopnoea index measured during laboratory polysomnography.
Limitations include the small number of participants, all covered by medical insurance, and the low number with significant comorbidity. Our results, therefore, may not be generalisable to all patients with suspected sleep apnoea, specifically those who are likely to have central sleep apnoea.
Another limitation was that 39% of the participants who attempted home testing had less than 4 h of valid recording on one or both of the home testing systems. However, the demographics of these participants were similar to those with successful tests.
In the last decade, many home testing devices have been introduced to diagnose sleep apnoea with the goal of reducing the inconvenience and expense associated with laboratory polysomnography. Most of these record several channels in addition to pulse oximetry. However, simple pulse oximetry is a low-cost and widely available test and in some clinical settings it is still recommended to screen for sleep apnoea.1–6 A guideline from the American Academy of Sleep Medicine states that “at a minimum, [portable monitors] must record airflow, respiratory effort, and blood oxygenation”.7 This means that home testing must be performed with a type III device, now usually using 4–7 channels, rather than a type IV device, as represented by an oximeter, recording only pulse rate and oxygen saturation. A recent publication from the American College of Physicians summarised validation studies with simultaneous recording of home testing systems during polysomnography and stated that “type III monitors have the ability to predict apnea-hypopnea index scores suggestive of OSA”.8 It added that “direct comparison between type III and type IV monitors was not possible” but “current evidence supports greater diagnostic accuracy with type III monitors than type IV monitors”.
Our study was designed to compare the ability of oximetry alone and of type III multichannel sleep polygraphy performed during unmonitored home testing to predict the apnoea–hypopnoea index (AHI) measured during in-laboratory polysomnography. For the type III system we chose the ResMed ApneaLink Plus (ResMed, San Diego, California, USA). A number of studies have shown good agreement between the AHI recorded with full polysomnography and the simultaneously measured AHI recorded by the ApneaLink, a single-channel nasal flow device.9–11 Other studies have tested two-channel and three-channel versions of the ApneaLink.12–14 The ApneaLink Plus is a type III device that measures respiratory effort, pulse oximetry and the pulse rate in addition to nasal airflow. While it has, to date, been assessed with only one validation study,15 it should be at least as accurate as the single channel nasal flow to predict the AHI since, according to current rules, a hypopnoea cannot be scored without an oxygen desaturation.16
Materials and methods
Adult patients scheduled for in-laboratory polysomnography with a diagnosis of suspected sleep apnoea were invited to participate with no exclusions for comorbidity. We attempted to do the home testing within 2 weeks of the polysomnography.
Stand-alone oximetry was performed with the Pulsox-300i oximeter (Konica Minolta, Inc, Osaka, Japan), which uses a finger probe connected to a recording module strapped to the wrist. The oxygen saturation and pulse rate were recorded as one sample per second. The oxygen desaturation index (ODI) was calculated with the Profox software (Profox Associates, Escondido, California, USA) using the criterion of a 4% decrease from the baseline to identify a desaturation event. In our data analysis, we used the ODI reported by the software with no technician intervention to exclude artefacts or portions of the record with a poor-quality signal. Tests were considered successful if there were at least 4 h of valid recording. The result of this test we call ODISOX.
The ApneaLink Plus records oxygen saturation and pulse rate with a finger probe, nasal airflow with a nasal pressure cannula and respiratory effort with a pneumatic sensor belt. The software reports the ‘apnoea–hypopnoea index’ or, according to current terminology, the respiratory disturbance index (RDI) using the 4% desaturation criterion and the ‘recommended rules’ of the American Academy of Sleep Medicine to score a hypopnoea.16 The test was considered successful if there were at least 4 h of valid recording on both oximetry and nasal airflow channels.
The ApneaLink Plus records were also manually scored after they were exported as European Data Format files into the Philips G3 Sleepware software (Philips Respironics, Carlsbad, California, USA). The records were scored by a sleep technologist blind to the results of the polysomnography, and in most cases the polysomnogram and the ApneaLink Plus records were scored by different technologists. Again we used a 4% desaturation and the ‘recommended rules’ to score a hypopnoea. We, therefore, reported four values from the ApneaLink Plus: the raw RDI and ODI from the device software (RDIRAW and ODIRAW) and the manually scored values (RDIMAN and ODIMAN).
Participants were instructed to put the two oximeter probes on different fingers of the same hand, but the choice of which probe was put on which finger was left to them.
The laboratory polysomnography was performed using the Compumedics Profusion Sleep 4 (Compumedics USA, Charlotte, North Carolina, USA) or the Philips Respironics Alice 5 systems. If a split-night test was conducted using part of the night for continuous positive airway pressure titration, only the diagnostic segment of the night was used for the analysis. Those tests that had been scored using the ‘alternative rules’ for a hypopnoea, requiring only a 3% desaturation, were rescored with the ‘recommended rules’ requiring a 4% desaturation.16
Statistical calculations were performed using GraphPad Prism V.5 for Mac OS X (GraphPad Software, San Diego, California, USA). A normal distribution of the data was not expected and so the non-parametric Spearman's rs was used for the tests for correlation. Bland-Altman plots were used to determine the biases between the AHI, and the several event rates that were calculated from the home tests and between RDIMAN and other estimates of RDI and ODI. We calculated receiver–operator curves to analyse the sensitivity and specificity of the home testing event rates to predict the AHI using cut-off values of 5, 10 and 15 events per hour. A second set of receiver–operator curves was calculated to test ODISOX, ODIRAW, ODIMAN and RDIRAW versus RDIMAN. RDIMAN was considered the 'best' of the home testing measurements.
One hundred and thirty-seven participants consented to the study (figure 1). Of these, 128 completed polysomnography. Seven cancelled the polysomnography or it was not approved by their insurers. Home testing was attempted by 123 participants, but the failure rate was unexpectedly high for both oximetry and the ApneaLink Plus. This may have been explained partly by the difficulty in keeping the two oximeter probes in place and perhaps by the participants not being highly motivated to get a successful study because they knew that they would be getting the ‘gold standard’ polysomnogram. Also contributing to the failures is the fact that we required that recordings be successful with both ApneaLink Plus and stand-alone oximeter. In addition, the ApneaLink Plus was ‘penalised’ by the requirement that we have 4 h of valid signals from both flow and oximeter channels. In the real world, the redundancy of flow and saturation channels would have allowed a clinically useful interpretation even if there were valid data from only one channel. Three participants were excluded from the analysis because they used the ApneaLink Plus and the stand-alone oximeter on different nights.
We were left with 73 participants who had at least 4 h of successful recording with both systems. The interval between home testing and the polysomnography was 2 weeks or less in 62 (85%) of participants (median 8 days, IQR 3–11, maximum 40).
Table 1 shows the demographic characteristics of these participants. Although we had a wide range of ages and body mass index, patients with comorbid conditions were under-represented. Specifically, we had no participants with a history of atrial fibrillation or of heart failure. The patients with sleep apnoea in our database of 737 with the condition were older, somewhat more obese and had more severe sleep apnoea (median age 58.0 years, body mass index 30.8, AHI 21.9) than the participants in this study. Twenty-three participants had an AHI of less than 5, 22 had 5–15 (mild), 18 had 15–30 (moderate) and 10 had more than 30 (severe sleep apnoea). Only three of our participants showed significant central events on the ApneaLink Plus (more than 5 central apnoeas per hour, and 50% or more of the RDI were central apnoeas).
The 50 participants who attempted home testing with technically unsatisfactory or no data on one or both systems are shown in table 2. They differed little in their demographics from those who were successful. However, their AHI was higher (p<0.01 by Mann-Whitney U test) and they included two participants with atrial fibrillation.
Tables 3 and 4 show the comparisons of the various calculations of the home testing respiratory event rates with the AHI measured during the laboratory polysomnogram. Only small differences were seen between the various home testing values and there was a large overlap of the confidence limits. Table 4 shows that, in terms of predicting a laboratory apnoea-hypoxia index above various thresholds, ODISOX and ODIRAW performed almost as well as RDIMAN. Specifically, comparison of the agreements of the AHI with RDIMAN and with ODIMAN shows that addition of the nasal flow and respiratory effort channels adds little to the ability of oximetry alone to predict the results of the laboratory polysomnography. Figure 2 shows the Bland-Altman plot for the laboratory AHI versus RDIMAN and ODIMAN. Figures 3 and 4 show the receiver-operator characteristic curves for RDIMAN and the three different values of ODI for cut-off values of the AHI of 5, 10 and 15. Spearman’s r and the receiver–operator characteristic areas under the curve suggest that RDIMAN had a slight advantage over the other home testing measurements to predict the AHI while the Bland-Altman plot shows that ODIMAN and ODISOX did slightly better.
In tables 5 and 6, we show the agreement between RDIMAN, the best predictor of the laboratory AHI, and the other event rates calculated from the home test. RDIMAN differed only slightly from RDIRAW and the three versions of the ODI.
This the first study that directly compares the ability of home testing with oximetry alone versus a type III device to predict the AHI based on laboratory polysomnography performed on a different night. There have been earlier studies in parallel groups comparing oximetry-based devices with polysomnography.17 ,18
There was only fair agreement between the results of the home testing and the in-laboratory polysomnography. Some of the sources of the poor agreement would be differences in sleep quality between the home and the laboratory environment, and the fact that the event rate was calculated for the EEG-based sleep time in the polysomnogram and for the recording time in the home tests. In addition, many of the polysomnograms were split-night studies in which slow-wave sleep might be over-represented and rapid eye movement sleep under-represented. However, the greatest source of the difference was probably night-to-night variation. For example, Levendowski et al19 reported a bias of 7.0 and a SD of 16.8 in a Bland-Altman plot of the AHI from two laboratory polysomnograms performed a month apart. Their SD was similar to what we found when we compared the laboratory AHI with the home testing event rate, measured on a different night, by any of the parameters we calculated. Chediak et al20 found that the AHI differed by 10 or more in 12 of 37 participants undergoing laboratory polysomnography on two successive nights.
However, our study was not intended to test the reliability of either home sleep polygraphy or stand-alone oximetry compared with laboratory polysomnography. Rather, we wanted to see if the addition of respiratory flow and effort channels to pulse oximetry significantly improved the ability of home testing to predict the AHI measured during an attended laboratory polysomnography. Our results suggest that it does not.
Given the currently accepted scoring rules, most respiratory events will be associated with a desaturation. The only events that would be scored differently by the ApneaLink Plus and by oximetry alone would be apnoeas without a desaturation and desaturations without an apnoea or a hypopnoea. Therefore, it is not surprising that, in the patients we studied, the RDI and the ODI agreed closely. Had we studied an older population with more comorbid cardiovascular or cerebrovascular disease our results may have been different; however, those are patients for whom home testing is not recommended.7
We did see some difference between the ODI reported by the ApneaLink Plus and the stand-alone oximeter. This could be explained in part by the different models of oximeter and the presumably different proprietary scoring algorithms used by the two systems. Also, there were some studies in which the valid recording time differed substantially between the two oximeters. So long as there were at least 4 h of valid signals for both, we included them in the analysis.
Validation studies using the ApneaLink and laboratory polysomnography simultaneously have shown that the manually scored RDI agrees better with the AHI than the RDI reported by the software.10 ,21 Our results show that RDIMAN agreed slightly better than RDIRAW with the laboratory AHI but, again, the differences were small compared with the confidence limits. Our Bland-Altman plots did not show the large differences between the manually scored and the computer-scored RDI for the ApneaLink Plus that were reported in a recent paper by Aurora et al.22 Unlike the 2014 paper from Masa et al,23 we did not see much difference between the receiver–operator characteristic curves for mild, moderate and severe sleep apnoea when comparing the laboratory polysomnography with home testing performed on a different night. However, they had studied a much larger group of participants than we did and they used the ApneaLink, which records only nasal airflow, rather than the four-channel ApneaLink Plus.
A 2014 guideline published by the American College of Physicians “recommends polysomnography for diagnostic testing in patients suspected of obstructive sleep apnea” but also “recommends portable sleep monitors in patients without serious comorbidities as an alternative to polysomnography when polysomnography is not available for diagnostic testing”.8 This paper reviewed the literature comparing type III and IV monitors with polysomnography and stated that direct comparison between the two types of monitors was not possible. However, it added that “indirect evidence from studies comparing each monitor with PSG suggested that type III monitors performed better than type IV monitors in predicting AHI scores suggestive of OSA”. Many type IV devices include other channels, such as pulse rate, but their scoring of respiratory events is based mainly on the ODI. Our study shows that if there is any advantage of the type III ApneaLink Plus compared with oximetry alone, the difference is small and probably clinically insignificant when compared with night-to-night variations between tests, regardless of the type of equipment used.
While sleep physicians may prefer the, probably slight, advantage of type III testing over simple oximetry for home testing of patients with suspected sleep apnoea, the low cost and availability of oximetry make it an attractive alternative in situations where the burden of undiagnosed sleep apnoea remains high. Even in the USA, underdiagnosis of sleep apnoea was a significant problem a decade or so ago, and it has probably not changed much since then.24
Although laboratory polysomnography provides much more information about a patient’s sleep than type III testing, its cost makes multiple night studies prohibitively expensive in most clinical situations. Some of the advantages of more detailed information are offset by the problem of night-to-night variations,19 ,20 ,25–27 which appears to be less with home testing.28–30 In the patient with no comorbidity and a high pretest probability of sleep apnoea, several nights of oximetry would certainly be less costly, and could be a more reliable diagnostic test than a single night of laboratory polysomnography. The high failure rate of home testing that we observed in this study was not observed in a large primary care case finding study and we believe it is largely explained by our requirement for 4 h of valid data from two oximeters plus the nasal flow channel of the ApneaLink Plus.31
Low-cost oximeters that can be used with a smart phone are readily available to the public and their use can be expected to increase rapidly with the growing popularity of ‘iHealth’ phone applications.32 The day is probably not far off when a pulse oximeter will be integrated into a wristwatch and there will be ‘apps’ that can display the ODI. More than three decades have passed since diagnostic testing for sleep disorders entered the main stream of clinical medicine and still we see educated patients with good health insurance presenting with severe and unrecognised obstructive sleep apnoea. A simple and self-administered test such as home oximetry may be required to get these people the help they need.
Wider availability of home oximetry can be expected to decrease one of the barriers to effective treatment of obstructive sleep apnoea by identifying undiagnosed patients. Even if type III devices provide a slight gain in sensitivity and specificity over stand-alone oximetry, the type of device used for diagnosis has little effect on the most important treatment outcome—the adherence to continuous positive airway pressure.33 ,34
The ability of home testing with the ApneaLink Plus to predict the apnoea–hypopnoea index measured during in-laboratory polysomnography hardly differed when it was based on either the software-calculated oxygen desaturation index or on the technician-scored respiratory disturbance index.
The differences between the oxygen desaturation index and the respiratory disturbance index measured during home testing are small compared with the reported night-to-night variation of the apnoea–hypopnoea index.
The authors are grateful to Kep Wadiak, David Parenteau and Sophie Pham for their skilled scoring of the polysomnograms and the home polygraphs. The ApneaLink Plus devices were provided by ResMed Corp. (San Diego, California, USA).
A preliminary report of this work was presented as a poster session at a meeting of the American Thoracic Society in San Diego, 19 May 2014.
Contributors AD designed the study, analysed the data and wrote a draft of the manuscript. RTL, SLA and DL contributed to the acquisition of data. DFK, LEK, RMG and DL contributed to the design of the study and the analysis and interpretation of data. All authors reviewed and commented on the manuscript and gave approval for the final version.
Funding This work was supported by the Scripps Health Foundation.
Competing interests None declared.
Patient consent Obtained.
Ethics approval Scripps Office for the Protection of Research Subjects (IRB-11–5671).
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.