Introduction

Pain is common in intensive care unit (ICU) patients, with an incidence up to 50% in surgical, as well as medical, patients [1, 2]. It has been shown that better management of pain in both intubated and non-intubated ICU patients, including a systematic evaluation of pain intensity and a therapeutic strategy of analgesic prescription, has been associated with improved outcome [3]. In this study, the median proportion of patients assessed using the self-administered Numerical Rating Scale (NRS) was 70% in non-intubated patients, contrasting with 100% of intubated patients assessed using the Behavioral Pain Score (BPS) administered by nurses. Indeed, pain assessment using a self-report scales (Visual Analogue Scale, NRS), as recommended in the general population [4–6], is not always possible in patients with altered neurological status.

There is no clinical tool to assess pain for non-intubated, non-communicating ICU patients, i.e., patients with delirium and/or an impaired vigilance status [7, 8]. To our knowledge, only the Critical Care Pain Observation Tool (CPOT) has been validated for mechanically ventilated and non-ventilated cardiac surgery patients [9]. This score has also been validated for mechanically ventilated, medical-surgical patients [10], but not for a mixed medical-surgical population of non-intubated ICU patients, and particularly not for ICU patients with delirium. Although similar, this four-domain score is distinct from the BPS, which is a score of only three behavioral domains, including facial expression, upper limb movements and compliance with ventilation. The BPS was initially elaborated to assess pain in nonverbal, mechanically ventilated patients with no severe head injury [11–13]. Because each domain of the BPS contains four descriptors instead of three for the CPOT, the BPS avoids a possible observer bias described when an observer rates preferentially the middle item of a three-point scale [14]. Use of the BPS is gaining interest in France and other countries [1, 13, 15].

The objective of this study was to construct and validate a new pain instrument devoted to non-intubated ICU patients (BPS-NI) unable to self-report their pain. We switched the “compliance with ventilation” domain of the initial BPS to a “vocalization” domain in this new form of BPS for non-intubated patients (BPS-NI). The choice of this domain was derived from Thunder Project II [16]. This study determined that vocal behavior was the most common pain behavior associated with the facial expression and the tonus of limbs. Vocal behavior was described as moaning, screaming, verbal complaints of pain and use of protesting words [16]. A vocalization domain of the BPS was then constructed and combined with the two other domains of the initial BPS.

Materials and methods

Detailed methods are provided in the Electronic Supplementary Material (ESM).

The present observational psychometric study took place in a 16-bed medical–surgical ICU. During a 7-month period, all consecutive patients ≥18 years old and staying in the ICU for more than 24 h were eligible if they were (1) non-intubated or non-trachetomized and (2) unable to self-report pain using a 0–10 enlarged NRS. This scale was adapted to ICU patients, who often suffer from sensorial deficiencies, by enlarging the printed scale to make it easily visible (10 × 30 cm) [3, 16]. Exclusion criteria were postoperative patients without any complications or organ dysfunctions, patients with severe brain injuries, quadriplegia, or history of severe dementia and mental retardation.

The scientific and ethics committee of the Comité d’Organisation et de Gestion de l’Anesthésie Réanimation du Centre Hospitalier Universitaire de Montpellier (COGAR) approved the design of the study. Because of the strictly observational study design and the absence of modification in clinical management of patients, the need for written consent from the patient or his relatives was waived.

Construction of the BPS adapted to non-intubated patients (see ESM)

The BPS evaluates three behavioral domains (i.e., facial expression, movements of upper limbs and compliance with ventilator). Each domain contains four descriptors that are rated on a 1–4 scale, and the total BPS value can range from 3 (no pain) to 12 (most pain) [11]. Training of nurses in the use of BPS has been evaluated several times in the unit for the reliability of their measurements [3]. The objective of the present study was to construct and validate a new tool, adapted from the original BPS to the non-communicant, non-intubated patient (BPS-NI). Like other pain scales, the BPS-NI can be used by caregivers to assess pain, for usual clinical practice or clinical research, several times a day, at rest and during nociceptive procedures. The procedure for using the BPS was estimated to take minimal time (2–5 min) [11]. Figure 1 shows the training poster of global BPS including the original BPS and the BPS-NI. The vocalization domain was described as “no pain vocalization,” “infrequent moaning (≤3/mn) and not prolonged (≤3 s),” “frequent moaning (>3/mn) or prolonged (>3 s),” and “howling or verbal complaints including Ow!, Ouch! or breath-holding.” Assessment of the duration (≤ or >3 s) of moaning seems to be common for reference pain caregivers because of the routine use of the Richmond Agitation Sedation Scale (RASS) in the ICU [17–19] (see below). In that scale, the observer assesses the level of vigilance by measuring objectively the duration of eye contact as < or >10 s [17–19].

Fig. 1
figure 1

Behavioral Pain Score training poster. This figure is a guide to training nurses and physicians to use the Behavioral Pain Score (BPS) in the ICU. The BPS, which was previously described and validated in non-communicating, mechanically ventilated patients, is extended in the present study to non-communicating, non-intubated or non-tracheotomized patients (BPS-NI). The first two domains are the same for the BPS and BPS-NI (i.e., facial expression and upper limbs movements). The third domain is different according to the mechanical ventilation status: compliance with ventilation (BPS) or vocalization (BPS-NI). The BPS and BPS-NI can be used by caregivers to assess pain in ICU non-communicant patients, for usual clinical practice or clinical research, like other pain scales, several times a day, at rest and during nociceptive procedures. The procedure for using the BPS was estimated to take minimal time (2–5 min) [11]. The ESM includes the original high-definition picture of this poster

Full size image

Study validation of the BPS-NI (see ESM)

Consecutive non-intubated patients were evaluated each morning by the bedside nurse for their ability to self-report pain with the NRS. If the patient failed to pass the test for two evaluations within a 4-h period, the bedside nurse contacted the pain referent physician (PRP) who attempted to evaluate the pain using the NRS. Patients able to rate their pain were not eligible for the study. The other patients, those who were unable to rate their pain, even with the assistance of the PRP (see ESM), were included in the study. The presence of delirium was checked by the PRP using the Confusion Assessment Method for the ICU (CAM-ICU) [20, 21]. We hypothesized that inattentiveness, disorganized thinking and/or an impaired vigilance status could explain, in part, the inability of the patient to use a 0–10 NRS. During the CAM-ICU procedure, the level of vigilance was measured using the RASS, which is the only vigilance scale validated in both ventilated and non-ventilated ICU patients [17–19]. Other reasons for the inability of the patient to self-report his/her pain included neurological and psychological disorders, such as impaired vigilance status, delusion, language disorders or incomprehension.

After neurological and psychological examination, the BPS-NI evaluation was independently performed within a working day at the same time by two paired evaluators (the PRP and bedside nurse) in four conditions for each patient: (1) at rest before and (2) during a non-nociceptive procedure (dressing change of a central venous catheter or an arterial catheter); (3) at rest-before and (4) during a nociceptive procedure (turning of the patient for the toilet and the massage of back and pressure points). This last procedure was considered the most common nociceptive procedure in the ICU setting [16]. For all of these measurements, the PRP was blinded to the BPS-NI values obtained by the other raters, i.e., the bedside nurses. Physiological parameters (heart and respiratory rates, mean arterial blood pressure and pulse oxymetry) were measured continuously and recorded by the PRP.

Statistical analysis (see ESM)

The validation of an instrument measuring a subjective variable (like pain) requires a comparison with a gold standard. In the absence of such a gold standard for non-intubated ICU patients who were unable to communicate, we had to validate the BPS-NI with indirect arguments, which consisted of checking the psychometric properties of validity, reliability and responsiveness according to standard definitions [22, 23] (see ESM). Methods of previous studies that validated the BPS in mechanically ventilated patients were used [11, 12]. The validity of the BPS-NI was tested in three ways (see ESM): by discriminative validity, internal consistency using the Cronbach α method [24] and factor structure by performing exploratory principal component factor analysis to determine the contribution of each item [25]. Only the BPS-NI evaluations performed by nurses were included for these analyses. Physiological parameters were analyzed by nonparametric tests. The inter-rater reliability of the BPS-NI was tested by the weighted kappa coefficient and by the correlation of the BPS-NI values observed by the nurses and the PRP, measured by the Spearman’s test (see ESM). The inter-rater agreement within an error of one mark was calculated as the ratio, expressed in percentage, between the number of the BPS-NI values different by more than one point between nurses and the PRP, and the total number of the BPS-NI paired values. Finally, the responsiveness of the BPS-NI was assessed by the effect size analysis [26] (see ESM).

Quantitative data were shown as medians and 25–75th percentiles. Significance for all statistical tests was set at P < 0.05. The sample size required for validation of the BPS-NI was established using the precision of a coefficient, such as Cronbach α [27]. Thus, with a precision of Cronbach α of 0.90 ± 0.05 as an objective, and for a value of three domains, it was required to include 30 patients in the study [12, 27].

Data were analyzed using the SAS software version 9.1 (SAS Institute, Cary, NC) by an independent confirmed statistician (GM).

Results

Among the 290 patients admitted to the ICU during the period of the study, 107 were excluded because they were postoperative patients without any complications or organ dysfunctions (n = 96) or died before extubation (n = 11). Among the 183 remaining patients who were not intubated or extubated during their ICU stay, 37 (20%) were unable to self-report their pain with the NRS. Of these, seven patients were excluded because of a history of mental retardation (n = 2), severe dementia (n = 2), stroke (n = 1), post-anoxic coma (n = 1) and cranial trauma (n = 1). Patient demographics are shown in Table 1. The impossibility for the patients to self-report their pain was mainly delirium (n = 25). The BPS-NI was tested by 18 of the 35 nurses and 15 of the 20 assistant nurses. All the nurses had several months of work experience in the unit. The 30 patients were evaluated for the four conditions during a working day. In all, 240 BPS-NI evaluations were performed. The 120 BPS-NI values measured by nurses were compared with the 120 BPS-NI values observed by the PRP.

Table 1 Demographics of the 30 patients included for analysis
Full size table

Validation study

The median BPS-NI value significantly increased from rest to nociceptive procedure [3.0 (3.0–3.8) vs. 6.0 (5.0–8.0), P < 0.001]. Contrary to the nociceptive procedure, the median BPS-NI value did not increase significantly during the non-nociceptive procedure [3.0 (3.0–3.8) vs. 3.0 (3.0–4.0), P = 0.11] (Fig. 2). These findings constitute a discriminative validation of the BPS-NI.

Fig. 2
figure 2

BPS-NI observed by nurses during the four conditions. This figure shows that the median Behavioral Pain Score adapted to non-intubated patients (BPS-NI), observed by nurses in the 30 patients included in the study, increased significantly from rest to a nociceptive procedure, but not from rest to a non-nociceptive procedure. This constitutes the validation structure procedure of the BPS-NI (discriminative validity). Medians are expressed as horizontal bars in bold, 25–75th percentiles as boxes and maximal–minimal values as vertical bars. NS = Non significant; ***P < 0.001

Full size image

Cronbach α values indicated that the BPS-NI had good internal consistency (raw Cronbach α = 0.77, standardized Cronbach α = 0.79), meaning that the three domains of the BPS-NI were well correlated between them.

Using exploratory principal component factor analysis, we found a large first factor, which accounted for 71% of the variance in pain expression, with a strong correlation of the domains with this factor, including coefficients of 0.59 for facial expression, 0.57 for upper limb movements and 0.57 for vocalization. In other words, this statistical method provided a mathematical and single surrogate value containing 71% of the information of the three domains of the BPS-NI and quantified the weight of each domain. These findings imply that all three domains of the BPS-NI were interrelated and reflected a pain expression factor in a balanced way.

Finally, there were slight, but significant, increases in RASS level, heart rate and respiratory rate during nociceptive procedures compared to other procedures (Table 2). However, a clinically relevant change of physiological parameters during nociceptive procedures was observed in less than 50% of patients (Table 2), whereas an increase of the BPS-NI of 2 or more points was measured in 25 patients (83%).

Table 2 Change of physiological parameters during the nociceptive procedures in the 30 patients included for analysis
Full size table

Reliability study

The weighted kappa coefficient, calculated to estimate the magnitude of agreement between the bedside nurses and the PRP, showed an important to near perfect agreement (all weighted kappa coefficients above 0.6) (Table 3). The magnitude of agreement remained important when only nociceptive procedures were taken into account, i.e., for BPS-NI values above 3 (Table 2). This agreement was not less important for the vocalization domain compared to the two other domains (Table 2). Within an error of one point, inter-rater agreement was 96% for the BPS-NI scores for both types of procedures and 90% for the BPS-NI scores for nociceptive procedures only. The correlation of the BPS-NI values between nurses and PRP was strong (r 2 = 0.88, P < 0.001) (Fig. 3). A nociceptive procedure BPS-NI score greater than 5 was measured by the nurses in 20 patients (67%) and by the PRP in 22 patients (73%).

Table 3 Magnitude of agreement for the Behavioral Pain Score-Non Intubated between bedside nurses and the pain referent physician measured by the weighted kappa test
Full size table
Fig. 3
figure 3

Correlation between the BPS-NI values observed by nurses and the pain referent physician. This figure shows a strong correlation between the Behavioral Pain Score adapted to non-intubated patients (BPS-NI) observed by nurses and BPS-NI observed by the pain referent physician (r 2 = 0.88, P < 0.001, Spearman’s test). Each number reflects how many results were observed per paired assessments

Full size image

Responsiveness study

The effect size for responsiveness was large for the three domains of the BPS-NI (facial expression = 2.82, upper limb movements 1.47, vocalization 3.64) and the total BPS values observed by the nurses (3.46).

Discussion

The BPS adapted to non-mechanically ventilated, non-intubated critically ill patients unable to self-report their pain (BPS-NI) is a valid, reliable and responsive instrument to measure pain in this population. The BPS-NI and the CAM-ICU could be used together to assess the patient’s pain and confusion, respectively.

A similar discriminative validity of BPS was shown in previous studies that measured the psychometric properties of BPS [11, 12], for which the mean value increased significantly from 3.0 at rest to 4.9 during a nociceptive procedure [11] and from 3.7 to 6.8 [12]. A similar large first factor was reported in previous studies [11, 12]. However, contrary to the two studies, which found a lower coefficient of correlation between the first factor and the domain of compliance with ventilation, in the present study we found a very well-balanced correlation between the first factor and all three domains. This could be explained by the modification of the third domain (i.e., “compliance-with-ventilation”), which was changed to “vocalization” in non-intubated patients. Compared to physiological parameters, the BPS-NI value changed more often during nociceptive procedures. Previous studies have shown that hemodynamic parameters were a poor surrogate to evaluate pain compared to behavioral tools [11, 12]. The complex pathology and important deregulation of physiology observed in critically ill patients could explain these findings [28].

The inter-observer reliability of BPS in the present and previous studies is high. Aïssaoui et al. [12] found an intraclass correlation coefficient of 0.95 and Payen et al. [11] a weighted kappa coefficient of 0.74. The very high coefficients measured in the previous and present studies, even taking into account only the nociceptive procedures, could be explained by the extensive training and experience of the ICUs in pain assessment. Finally, the responsiveness of the BPS was previously measured using the effect size coefficient, which ranged from 2.2 to 3.4 for the total BPS value [12], similar to the results of the present study (3.46).

Since the review of pain measurement instruments available in the ICU setting by Hamill-Ruth and Marohn 10 years ago [7], which highlighted the absence of validated instruments for the critically ill patients who are often unable to communicate, aside from the COMFORT scale for children [29], several new behavioral pain instruments have been described in the literature [9, 11, 16, 30–33]. However, a recent review [8] of all these instruments concluded that only the BPS [11] and the CPOT [9] have been shown to provide acceptable levels of validity and reliability. Aside from the BPS-NI validated by the present study, to our knowledge, no pain measurement instruments have been evaluated for use in non mechanically ventilated medical-surgical ICU patients with delirium and unable to self-report their pain.

Our study has several limitations. The main limitation is that the validation study used an indirect method to assess pain in the absence of an objective pain measurement reference. Secondly, as in similar studies [11, 12, 17, 18, 34, 35], it was impossible to blind nociceptive and non-nociceptive conditions. Thirdly, because of the aim of this study, performed in a single center with a small number of patients, was to measure the psychometric properties of the BPS-NI, further studies are needed to show the transferability of this tool to other teams and the clinical impact of the use of such instruments, as this would probably constitute the most pertinent validation of an instrument. To our knowledge, only one study has demonstrated positive clinical results through use of the BPS to manage pain in ICU patients at rest [3]. However, 30% of scheduled assessments of pain were not performed in non-intubated patients. The absence of an available pain measurement instrument for non-intubated, non-communicating patients is likely the cause for this lack of assessment. This study’s findings may have been more pertinent if a “universal” BPS had been used, such as the BPS-NI of the present study, in conjunction with the original one. The moderate use of at least World Health Organization step 2 or more, or similar effect analgesic drugs observed in the present study at rest (23%), could be greater, such as their use during nociceptive procedures, after the implementation of a BPS-NI based analgesia protocol.

Finally, although we included mainly patients with delirium, we did not validate this score in patients with a large impaired vigilance status. However, it is likely that many of these patients, for example, comatose patients, would require intubation.

In conclusion, the Behavioral Pain Scale-Non Intubated (BPS-NI), an adaptation of the original BPS for non-mechanically ventilated critically ill patients who are unable to self-report their pain, such as patients with impaired vigilance status and/or delirium, is a valid, reliable and responsive instrument to measure pain in this population of ICU patients. This pain scale could be used by caregivers several times a day to assess pain for usual clinical practice and clinical research, at rest and during nociceptive procedures. Further studies are needed to measure the clinical impact of the use of this instrument upon improvements in pain management.