Introduction

Blood glucose control aiming at normoglycemia (i.e., blood glucose levels 80–110 mg/dl), frequently referred to as “tight glycemic control”, has been shown to reduce mortality and morbidity of critically ill patients [1, 2] although its implementation bears the risk of hypoglycemia [1, 2]. Applying tight glycemic control (TGC), however, requires additional efforts, especially from the nursing staff who are expected to adhere to the TGC protocol. Such protocols are often complex, requiring specific timing of blood glucose level (BGL) measurements, and the availability of patient-specific data. One way to support nurses in adhering to protocols is by applying information technology, in particular clinical computerized decision-support systems (CDSSs).

In general, a CDSS is a computer program that is intended to help healthcare workers in making decisions [3]. A CDSS can be characterized by the level of support, the consultation mode, and the communication style. The level of support ranges from general to patient-specific: e.g., from merely displaying the protocol chart to suggesting the specific amount of insulin to be administered. Some systems are passive, providing advice only on demand, whereas others are active, providing feedback to the healthcare worker without being asked for it. Finally, regardless of the level of support and the consultation mode, a CDSS may operate in two communication styles: in the critiquing mode the system provides advice which is dependent on the adherence of clinical practice to a protocol (e.g., notifying the nurse that a BGL measurement has been expected but not performed), in the non-critiquing mode it provides advice regardless of whether a protocol is followed or not [4]. Notwithstanding the potential benefits of CDSSs, issues pertaining to their design, implementation, evaluation, and to critical success factors, are largely still open. Glucose regulation is specifically interesting as it forms an application in which a CDSS is applied in a highly controlled clinical practice. Therefore lessons learned from glucose regulation CDSSs can be used for other highly controlled clinical practices, for example mechanical ventilation and blood-pressure control.

The objective of this systematic review is to identify and summarize characteristics of CDSSs and to review their effects on the quality of the TGC process in published studies on the use of CDSS in TGC for critically ill patients.

Methods

We searched for relevant English language articles based on keywords in title, abstract and MeSH terms, using Ovid Medline and Ovid Medline In-Process (1950 to December 31, 2008). The final literature search was performed on January 11, 2008.

Figure 1 shows the applied search strategies and the corresponding search flowchart. In the first stage (A), we searched for terms related to glucose and insulin. In the second stage (B) we limited the search using the terms “critical illness” or “critical care” or “intensive care”. In the third stage (C), keywords and MeSH terms referring to a decision-support system were searched. The results of these three stages were combined using the Boolean operator “AND”. Searching was supplemented by scanning bibliographies from identified articles.

Fig. 1
figure 1

The search strategies applied and the search flowchart. The bold terms are MeSH terms

Full size image

Two reviewers independently examined all titles and abstracts. Discrepancies among the two reviewers were resolved by consensus involving a third reviewer. Articles were selected if they reported original data from a clinical trial or observational study on critically ill adult patients and only if one of their main objectives concerned the evaluation of a given TGC protocol with a CDSS. A study was included if the TGC protocol implied an upper normoglycemia limit of, at most, 150 mg/dl. Opinion papers, surveys, and letters were excluded.

From the selected papers, the same two reviewers extracted data on the following items:

  1. 1

    method and study design aspects;

  2. 2

    CDSS characteristics;

  3. 3

    TGC quality indicators in terms of their definition and applicability; and

  4. 4

    data on the effect of the CDSS on the quality of the TGC process or mortality or morbidity.

A quality indicator was defined as a measurable quantity of the TGC process that may, alone or in combination with other quantities, indicate some aspect of its quality. Discrepancies between the two reviewers were again resolved by consensus after involving the same third reviewer.

To obtain insight into the heterogeneous nature of these evaluation studies and according to the hierarchy of study designs developed by the University of California San Francisco Stanford Evidence-Based Practice Center we classified the studies into:

  • randomized controlled trial (level I),

  • non-randomized controlled trial (level II),

  • observational study with controls (level III), and

  • observational study without controls (level IV) [5, 6].

Results

Searching the online databases resulted in 70 articles. Initial screening of titles and abstracts rendered 21 articles eligible for further full-text review. One additional article was identified by reviewing bibliographies, yielding a total of 22 articles. Based on the full text review, five studies were excluded because they turned out not to address a computerized system, leaving 17 articles for detailed analysis.

Table 1 lists the materials (included patients and study locations), TGC target range, design, and results of these 17 studies. Table 2 summarizes the CDSSs’ characteristics. Table 3 reports all the used quality indicators of the TGC process.

Table 1 Description of study design, TGC target range, and results of the 17 included papers on DSS for TGC implementation
Full size table
Table 2 Characteristics of CDSS
Full size table
Table 3 List of quality indicators used
Full size table

Methods and study design aspects

There were three randomized clinical trials, seven controlled before/after trials, and seven observational studies. In two studies only information was given on how often the CDSS was used, in one of these studies this information was only given for the three months after the study period.

In only one study the same TGC protocol was used before and after the implementation of CDSS. In five other studies a modified version of the TGC protocol was used after the implementation of the CDSS. Two of these five studies evaluated the same CDSS and the protocol used, but in another setting. Eleven out of seventeen studies evaluated the effect of a new TGC protocol introduced at the same time with the implementation of the CDSS. Three of these eleven studies also evaluated the same CDSS and the protocol in different settings.

CDSS characteristics

Most of the reported CDSSs (14 out of 17) were stand-alone and were not integrated in other clinical information systems, for example computerized physician order entry systems, patient data management systems, or intelligent pumps. CDSSs in all studies were “patient specific” and operated in the “critiquing” mode. This means that decision support was given if clinical practice for a patient was not according to the protocol. Reminders on the time of the next BGL measurement were “active” in nine out of seventeen studies. “Active” means that the users automatically received the reminder without manually asking for it. Support for insulin pump speed was “active” in one study only; the others used the “passive” mode for pump adjustments requiring the clinician to ask for advice.

In five studies the protocols were based on “if–then” statements on a sliding scale. They contained a list of simple rules, with a condition (the “if” part) and a conclusion part (the “then” part). The condition was based on the value of the current BGL measurement. The conclusion specified the corresponding insulin amount (or insulin pump speed) and the time interval until the next BGL measurement. In the other 12 studies the protocols were formula-based. Formula-based protocols rely on an familiar and simple equation: insulin dose/hour = [BGL − 60] × multiplier (insulin sensitivity) [7], which was sometimes adapted. In this group of studies, a default number for the multiplier was considered as the starting point for TGC. Based on the latest BGL values and a statistical model, the multiplier was recalculated and these values were used to calculate the required next pump speed.

In most studies (14 out of 17), users manually entered the BGL values and pump speeds into a separate CDSS database, because these data were not electronically available or they were not connected to the CDSS. In three other studies these data were electronically retrieved from a laboratory or a hospital information system.

Quality indicators

Twenty-four different indicators of glycemic control were extracted (Table 3). Hypoglycemia-related indicators were used in 14/17 studies as a proxy for safety. Six different thresholds varying between 40 and 70 mg/dl were used to define a hypoglycemia event. The most often used indicators were BGL summaries such as mean or median BGL (15/17 studies), the number of measurements in a predefined target range (10/17), the frequency of BGL measurements (9/17), the time needed to reach the defined BGL target (8/17), the time spent in the predefined BGL range (7/17), or compliance to protocol (6/17). Hyperglycemia-related indicators (number of hyperglycemia events and hyperglycemia index) (6/17) were other frequently used indicators. Because of different target ranges, comparing these indicators is difficult. Lower limits of target ranges varied from 80 to 100 mg/dl. Similarly, the upper limits of target ranges varied from 110 to 150 mg/dl.

Effect of the CDSS on the quality of the TGC process

All controlled studies reported on at least one quality indicator that was improved by introducing the CDSS (with or without the new protocol). Among controlled studies, one study reported that the number of hypoglycemia events increased, but without mentioning whether this increase was statistically significant. Among other controlled studies one study did not report [8], six studies reported no change, and two studies [9, 10] even reported a reduction in the number of hypoglycemia events. Seven observational studies also reported that the number of hypoglycemia events was in an acceptable range. No clinical symptoms attributable to hypoglycemia were reported in any of the studies. The observational studies also used efficiency-related indicators, for example mean BGL or time to capture TGC in the acceptable range by introducing the CDSS (with or without a new protocol).

Discussion and recommendations

In this review we have summarized the design, characteristics, indicators, results, and limitations of 17 published studies on glucose regulation CDSSs in intensive care. Although most studies reported a positive effect on at least one quality indicator the diversity of studies in term of case-mix, insulin therapy, associated therapies, and indicators used (varying in their definition and the ways of calculating them) severely hamper comparison of the studies. Therefore, although meta-analysis is theoretically possible, the results will not be reliable. In addition only three papers reported on an RCT. Our search included all synonyms known to us of TGC, for example “intensive insulin therapy” (IIT). A limitation of our search is that we only addressed studies whose main objective concerned evaluation of a given TGC protocol with a CDSS; we might have missed some studies with a limited evaluation and TGC quality measurement focus.

The implementation of paper-based TGC protocols with decision tables or charts for adjusting the insulin rate is cumbersome and time-consuming [11]. Although for many studies without CDSS implementation results were acceptable [12, 13], frequent measurements are crucial in TGC. Furthermore, a paper-based TGC protocol cannot remind the users about the time for the next measurement. For these reasons, CDSSs may improve glycemic control and help implementing TGC. To the best of our knowledge this is the first review exclusively dedicated to supporting the TGC protocol with a CDSS. Existing reviews on TGC focused on the range of TGC, the effects, and advantages, regardless of the implementation strategy of the TGC protocol, such as a CDSS [12, 13]. The remainder of this section will discuss the findings on study design, CDSS characteristics, and TGC quality indicators and will provide recommendations on these subjects for future research.

Study design

Most of the studies used a before–after design and introduced the CDSS together with a new protocol. Therefore, it is difficult to conclude that the CDSS itself was the causative reason for observed improvements in glucose regulation. The fact that in some studies the users differed before and after the introduction of the CDSS (based on a new protocol) further hampers identification of success factors for improved glucose regulation. Future research should carefully choose a study design in order to clearly separate the effect of the TGC protocol and the contribution of the CDSS.

CDSS characteristics

To achieve the optimum effect of TGC, the protocol should be integrated into clinical workflows [14]. It should put the knowledge for clinical decision-making at the point of care. The question is, however, what is the best place and time for presenting the knowledge to the user. Boord et al. [15] used the computerized physician order entry (CPOE) as the starting point and integrated the CDSS into this. They believe this provided a “one stop shop” by capturing BGL and other patient data from the clinician at the bedside, generating new orders, and logging the data into the electronic medical record. Clinicians should first initiate an order in the CPOE before the CDSS could generate advice. Both pump adjustment and next measurement time reminders were passive. In contrast, Vogelzang et al. [16, 17] believe that because most nurses spend some time near the glucose analyzer until the result of the analysis is known, a computer situated next to the analyzer is the ideal spot for the system to interact with the nurses. This system generated an active reminder but it could be shown only when nurses were near the analyzer, missing other clinicians. As a final example, Rood et al. [8] evaluated a system which got the necessary data from a patient data-management system (PDMS), generating reminders during care at the bedside computer, and logging the data into the electronic medical record. This system provided an active reminder but it is unknown what happens with a reminder if the user was logged out or was away from the computer. Altogether more research is necessary to investigate the most appropriate implementation site, target user, and time of advice. This will also depend on local circumstances such as technical infrastructure and responsibilities of TGC implementation.

TGC quality indicators

We found no uniform indicator set of glycemic controls was used in the studies reviewed. Most indicators differed in their definitions among the studies although they are all meant to measure the same underlying concept. For example the sampling frequency of measurements plays a crucial role in TGC but it was reported in nine out of 17 studies. In these nine articles, six different ways were used to calculate the sampling frequency (e.g., mean or median per patient or per patient treatment day or mean sampling interval). Thus reproducibility and comparability of research results are hampered by this lack of unambiguous definitions. The choice of quality indicators used was not explicitly mentioned in any of the studies. We could not find an association between indicator selection and patient population, diseases, or specification of the designed protocols.

Because hypoglycemia is the main risk of TGC implementation, almost all studies reported on at least one indicator related to hypoglycemia. The number of hypoglycemic events before and after TGC implementation and/or the management of these events form the main safety-related indicators of TGC. However, we found several definitions and ambiguous terminology for rendering a BGL measurement (or a set thereof) as a hypoglycemic event. Most of the studies reported no change or even a reduction in the number of hypoglycemia events. This finding is in contrast with some studies, including a recent meta-analysis [18], showing that TGC implementation increased the risk of hypoglycemia events. TGC, semantically, excludes the occurrence of hypos because it is “tight”. Thus, if one is applying a TGC protocol but hypoglycemia occur then the culprit resides in the implementation of the regulatory process. CDSSs are used to improve adherence to TGC protocols; therefore, if the protocol is sound a CDSS contributes to keeping BGL in range and hence to reducing the risk of hyperglycemia and hypoglycemia.

The strong relationship between hyperglycemia and mortality and morbidity is well known from the literature. Hence hyperglycemia reduction is the main objective of TGC. Surprisingly, only six studies explicitly defined a hyperglycemic event and/or hyperglycemia index, but they used different definitions of such an event.

The increased number of blood-glucose measurements, especially in well regulated patients, may increase the number of measurements with low glucose levels and may thus reduce the mean and median plasma glucose level of the whole group, giving a false impression of improvement of the studies.

There is almost no literature that compares different glycemic metrics to relevant clinical outcomes, such as severity-associated mortality [19]. It is, therefore, hard to evaluate and qualify these indicators. Deciding upon a common glycemic vocabulary is hence essential.

Applying the results of highly controlled clinical trials like those evaluating TGC to everyday practice is difficult [20]. The need for frequent measurements and pump adjustments increases the need for CDSSs. If a passive CDSS is implemented, we cannot be sure that the users ask for support on time. In such a case usage of the system (frequency) should be clearly described. If an active CDSS is implemented based on the content of these highly controlled protocols, the reminders will be shown many times. Showing many pop-up reminders usually irritates users, resulting in the alerts being ignored [5, 6]. Also, in an active CDSS it is not clear how many times the messages are actually seen on time and not too late, because of users being logged off or away from the computer. Therefore, in future research it is necessary to report the usage, usability, and satisfaction with such CDSSs.

Data verification is very important, especially when the values are entered manually in information systems. Button et al. [21] implemented double data entry as a data-verification tool. Morris et al. [22] mentioned this issue, although they did not implement any data verification techniques. They are of the opinion that enabling bedside nurses to use CDSS with minimum effort outweigh the theoretical advantage of additional security and double data entry. Similar to others, in their study the accuracy of manually entered values and the possibility of introducing new kind of errors were not reported or discussed. Using CDSSs without a data-verification process might be a critical safety issue and hence authors ought to address this issue when discussing their results.

Conclusion

While most studies evaluating the effect of CDSS on the quality of the TGC process found improvement when evaluated on the basis of the quality indicators used, it is impossible to define the exact success factors. This is mainly because of the lack of standard agreed-upon indicators of glycemic control, the simultaneous implementation of the CDSSs with new treatment protocols, and the various solutions used for integrating the CDSS into the users’ workflow. This systematic review provided key recommendations and information for researchers and ICU managers who want to develop and evaluate CDSSs for glucose regulation or other highly controlled clinical practices.