Intended for healthcare professionals

Endgames Statistical Question

Bias in observational study designs: cross sectional studies

BMJ 2015; 350 doi: https://doi.org/10.1136/bmj.h1286 (Published 06 March 2015) Cite this as: BMJ 2015;350:h1286
  1. Philip Sedgwick, reader in medical statistics and medical education1
  1. 1Institute for Medical and Biomedical Education, St George’s, University of London, London, UK
  1. p.sedgwick{at}sgul.ac.uk

Researchers investigated the association between body mass index (BMI) and both sexual behaviour and adverse sexual health outcomes, as well as their importance in obese people. A telephone survey with a cross sectional study design was used. Participants were French speaking men and women aged 18-69 years who lived in France. The telephone survey was a population based one that involved a random sample of households selected from the national telephone directory. In total, 12  364 people were contacted by telephone between September 2005 and March 2006. Of those initially selected, 10 170 (82.3%) agreed to complete the questionnaire.1

An association between BMI and both sexual behaviour and adverse sexual health outcomes was reported. In particular, obese women were less likely to access contraceptive healthcare services and were more likely to have an unplanned pregnancy. It was concluded that the prevention of unintended pregnancies among obese women is a major reproductive health challenge. Healthcare professionals need to be aware of sensitivities related to weight and gender in the provision of sexual health services.

Which of the following statements, if any, are true?

  • a) The use of a population based study minimised selection bias

  • b) The sample would have been prone to non-response bias

  • c) The sample was prone to volunteer bias

  • d) The results of the study were prone to ascertainment bias

Answers

Statements a, b, c, and d are all true.

The researchers performed a cross sectional study, which is observational in design. Cross sectional studies have been described in a previous question.2 In an observational study the investigators do not intervene in any way but record the health, behaviour, attitudes, or lifestyle choices of the study participants. As the name suggests, the aim of a cross sectional study is to obtain a representative sample by taking a cross section of the population. All the measurements for a study participant are obtained at a single point in time, although recruitment may take place across a period of time. Cross sectional studies are generally quick, easy, and cheap to perform, and they often involve a questionnaire survey.

In the study above, the participants were French speaking men and women aged 18-69 years who lived in France. A telephone survey was conducted, involving a random sample of households selected from the telephone directory. Each participant was interviewed once, with recruitment taking place between September 2005 and March 2006. As for any study, the method of sampling and availability of potential participants will have affected the extent of selection bias. Selection bias is a general term used to describe a group of biases and effects that result in a sample that is systematically different from the population it is intended to represent. The study above was population based—that is, the sample was obtained from a defined population. A sample resulting from a population based study is therefore likely to be representative of the population, thereby minimising selection bias (a is true). The use of random sampling across the population meant that selection bias was minimised further. This is in contrast to a sample from, for example, a hospital clinic using convenience sampling3—such a sample would probably be systematically different from the population with respect to its demographics and health outcomes. If selection bias exists it results in a lack of external validity—that is, the extent to which the study results can be generalised to the population that the sample is meant to represent.

In the study above, participants were contacted using random sampling of households from the national telephone directory. Therefore, ownership of a phone and listing in the directory would have influenced inclusion in the study. Furthermore, people would have been included in the study only if they were at home to answer the telephone call if randomly selected. The extent to which these factors would have influenced selection bias is difficult to quantify; limited information would have been available for those people who did not own a telephone or for those who were not at home when the researchers called.

In total, 12 364 people were contacted by telephone, of whom 10 170 (82.3%) agreed to complete the questionnaire. Therefore, the resulting sample would have been prone to non-response bias (b is true). Non-response bias would have occurred if those people who did not accept the invitation to be part of the study were different from those who did. However, any differences in characteristics (including demographics, BMI, and sexual behaviour plus adverse sexual health outcomes) would have been difficult to quantify because limited information, if any, would be available for those who were selected but refused to be part of the study. Non-response bias is a particular problem for questionnaire surveys because usually not all people who are approached will agree to take part. Non-response bias should not be confused with volunteer bias or response bias, which are described below.

The sample in the study above was prone to volunteer bias (c is true)—a systematic difference between those people who volunteered to be part of the survey and the population. The volunteers would be expected to differ from the population in their sociodemography, behaviour, attitudes, and health. It has been reported that, in general, those who participate in studies are more educated, come from a higher social class, and are more sociable than those who do not participate. Non-response bias and volunteer bias are often confused. Non-response bias focuses on the potential differences between the non-responders and responders originally selected for the sample, whereas volunteer bias considers the potential differences between those who volunteer and the population. Both will result in selection bias.

The collection of data in the above study was prone to ascertainment bias (d is true). Such bias would have occurred if the information recorded for the respondents regarding their BMI, sexual behaviour, or adverse sexual health outcomes was systematically different from their actual BMI or experiences. Such bias in data collection can be unconscious or otherwise, and can originate from the investigators or participants. When ascertainment bias occurs on behalf of the participants, it is referred to as response bias. It is a particular problem in questionnaire surveys that investigate socially unacceptable or potentially embarrassing behaviours. For example, participants may have under-reported their weight or sexual behaviour because of embarrassment. If information bias occurs on behalf of researchers or interviewers it is referred to as assessment bias, sometimes known as observer bias. In the above study assessment bias might have occurred despite the telephone interviewers having been trained. The recording of data could have been influenced by the attitudes or past experiences of the interviewers, particularly if the participants provided responses that were subjective and difficult to qualify or quantify.

Cross sectional studies are sometimes repeated to assess trends over time. However, if different participants are included at each time point then caution will be needed when interpreting the results. It may be difficult to assess whether changes in behaviour or attitudes reflect a trend or simply differences between different groups of participants sampled from the population.

Notes

Cite this as: BMJ 2015;350:h1286

Footnotes

  • Competing interests: None declared.

References

View Abstract