Double Data Entry: What Value, What Price?

Abstract

We challenge the notion that double data entry is either sufficient or necessary to ensure good-quality data in clinical trials. Although we do not completely reject that notion, we quantify some of the effects that poor quality data have on final study results in terms of estimation, significance testing, and power. By introducing digit errors into simulated blood pressure measurements we demonstrate that simple range checks allow us to detect (and therefore correct) the main errors that impact the final study results and conclusions. The errors that cannot easily be detected by such range checks, although possibly numerous, are shown to be of little importance in drawing the correct conclusions from the statistical analysis of data. Exploratory data analysis cannot identify all errors that a second data entry would detect, but on the other hand, not all errors that are found by exploratory data analysis are detectable by double data entry. Double data entry is concerned solely with ensuring, to a high degree of certainty, that what is recorded on the case record form is transcribed into the database. Exploratory data analysis looks beyond the case record form to challenge the plausibility of the written data. In this sense, the second entering of data has some benefit, but the use of exploratory data analysis methods, either as data entry is ongoing or at the end of data entry and as the first stage in an analysis strategy, should always be mandatory.

Introduction

Few people would dispute the notion that good-quality data are important and certainly preferable to poor-quality data. Although we concur with this notion, we cannot explicitly state what we mean by “good” or “bad” data quality, nor can we specify whether the quality of any specific dataset is adequate for the intended analysis. The better the quality of the data, the more reliable and convincing the inferences will be; thus, on any arbitrary scale that measures quality, all we can really say is that better-quality data lead to sounder and more reliable conclusions. Whether quality is already good or bad, moving along that scale to improve quality seems desirable. We do not have many ideas about how much quality matters despite the fact that quality is often simple to measure.

Many steps are involved in getting data from a patient into a final report [1]. Each may introduce a variety of errors. We believe that building quality into systems is more productive than building checks onto the end.

Despite the many points data pass on their route from patient to final report, data entry may be the only one where, quite routinely, we process everything twice. A few studies have duplicate investigators' assessments; none is likely to have duplicate case record forms (CRFs) to catch transcription errors; the concept of typing a final report twice to check for typographical errors is almost laughable. Rarely, two independent statisticians may analyze the data or the statistician may use more than one software package. So why double data entry but not double everything else? We return to this question later in the discussion.

Although double data entry is useful for identifying typographical errors, we have no objective data on its effectiveness. Some of the differences (or errors) it identifies may be of interpretation; others may be simple typing errors, but we cannot easily determine the number of errors that are not identified by double data entry. What effect do these errors have? Can errors be found in other ways? Does double data entry add value or waste time? “What is of interest is not the error rates themselves but whether the conclusions differ before and after correcting the errors” [2].

Others who have considered some aspects of this question have discussed what is achievable or what types of error rates result from different data entry procedures. Reported achievable rates vary. Neaton et al [3]cites 10 per 10,000 fields; Shenker [4]cites 7 per 10,000 keystrokes (in a project involving 350 million keystrokes); and Jeffreys (see Hosking et al [5]) reported that in a given study, error rates fell over time from 10 to between 4 and 8 per 10,000 fields.

None of these authors, however, addresses whether those errors mattered. Hilner et al [6]describe reviewing dietary data and report critical errors at 0.12%. They define critical errors as “those impacting on nutrient computations.” Such errors might be specifically related to the primary outcome but still not critical in the sense of affecting the conclusions drawn from the study. Other papers have discussed assessments of data quality [7]and objectivity in measuring data quality [8]. An example of objectivity is whether to regard a date as one or three fields. Are errors in the day and month, for example, two errors or simply one? Additional material has been published that directly compares, in an experimental setting, single versus double data entry [9]. Seaman (without specifying single or double data entry) notes that databases have sometimes completely missed some data on the CRF [10].

This article uses an experiment to consider the value of double data entry. We assume that data entry errors occur at random, and that this affects each treatment group similarly.