Skip to main content
SpringerLink
Log in

Assessing the quality of diagnostic studies using psychometric instruments: applying QUADAS

  • ORIGINAL PAPER
  • Published:
Social Psychiatry and Psychiatric Epidemiology Aims and scope Submit manuscript

Abstract

Background

There has been an increase in the number of systematic reviews of diagnostic tests, which has resulted in the introduction of two checklists: statement for reporting of diagnostic accuracy (STARD) and quality assessment of diagnostic accuracy studies (QUADAS).

Objective

To examine the validity and usefulness of QUADAS when applied to diagnostic accuracy studies using psychometric instruments and to examine the quality in reporting of these studies during practical application of the checklist.

Method

Two reviewers independently rated the quality of 54 studies using QUADAS. The proportion of agreement was used to assess overall agreement and individual agreement of QUADAS items between reviewers.

Results

The overall agreement between the two reviewers for all QUADAS items combined was 85.7%. The proportion of agreement between reviewers for each item ranged from just over 57–100% and was over 80% for 8 of the items. The poorest agreement was associated with the items for selection criteria, indeterminate results and withdrawals. None of the studies adequately reported all relevant information to enable all QUADAS item to be scored as ‘yes’.

Conclusion

Overall QUADAS was relatively easy to use and appears to be an acceptable tool for appraising the quality of diagnostic accuracy studies using psychometric instruments. The application of QUADAS was hampered by the poor quality of reporting encountered.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Abiodun OA (2006) Postnatal depression in primary care populations in Nigeria. Gen Hosp Psychiatry 28:133–136

    Article  PubMed  CAS  Google Scholar 

  2. Adewuya AO, Eegunranti AB, Lawal AM (2005) Prevalence of postnatal depression in Western Nigerian women: a controlled study. Int J Psychiatry Clin Pract 9:60–64

    Article  Google Scholar 

  3. Agoub M, Moussaoui D, Battas O (2005) Prevalence of post partum depression in a Moroccan sample. Arch Womens Ment Health 8:37–43

    Article  PubMed  CAS  Google Scholar 

  4. Altman DG, Schulz KF, Moher D, Egger M, Davidoff F, Elbourne D, Gøtzsche PC, Lang T (2001) The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 134:663–694

    PubMed  CAS  Google Scholar 

  5. Areias M, Kumar R, Barros H, Figueiredo E (1996) Comparative incidence of depression in women and men, during pregnancy and after childbirth: Validation of the Edinburgh postnatal depression scale in Portugese mothers. Br J Psychiatry 169:30–35

    Article  PubMed  CAS  Google Scholar 

  6. Aslan E, Kuey L, Kultur S (1997) The validity and reliability study of the Turkish version of the Edinburgh postnatal depression scale. In: Spring Symposium–1 Book. Psychiatric Association of Turkey, Ankara

    Google Scholar 

  7. Austin MP, Hadzi-Pavlovic D, Saint K, Parker G (2005) Antenatal screening for the prediction of postnatal depression: validation of a psychosocial Pregnancy Risk Questionnaire. Acta Psychiatr Scand 112:310–317

    Article  PubMed  Google Scholar 

  8. Aydin N, Inandi T, Yigit A, Nalan Sahin Hodoglugil N (2004) Validation of the Turkish version of the Edinburgh postnatal depression scale among women within their first postpartum year. Soc Psychiatry Psychiatr Epidemiol 39:483–486

    Article  PubMed  Google Scholar 

  9. Bagedahl-Strindlund M, Borjesson KM (1998) Postnatal depression: a hidden illness. Acta Psychiatr Scand 98:272–275

    Article  PubMed  CAS  Google Scholar 

  10. Barnett R, Matthey S, Gyaneshwar R (1999) Screening for postnatal depression in women of non-English speaking background. Arch Womens Ment Health 2:67–74

    Article  Google Scholar 

  11. Beattie-Clarke P (2003) Validation of two postpartum screening scales in a sample of Saskatchewan First Nations and Metis women. In: Psychology. University of Regina

  12. Beck CT, Gable RK (2001) Comparative analysis of the performance of the postpartum depression screening scale with two other depression instruments. Nurs Res 50:242–249

    Article  PubMed  CAS  Google Scholar 

  13. Beck CT, Gable RK (2001) Further validation of the postpartum depression screening scale. Nurs Res 50:155–164

    Article  PubMed  CAS  Google Scholar 

  14. Beck CT, Gable RK (2005) Screening performance of the postpartum depression screening scale—Spanish version. J Transcult Nurs 16:331–338

    Article  PubMed  Google Scholar 

  15. Benvenuti P, Ferrara M, Niccolai C, Valoriani V, Cox JL (1999) The Edinburgh postnatal depression scale: validation for an Italian sample. J Affect Disord 53:137–141

    Article  PubMed  CAS  Google Scholar 

  16. Berle JO, Aarre TF, Mykletun A, Dahl AA, Holsten F (2003) Screening for postnatal depression. Validation of the Norwegian version of the Edinburgh postnatal depression scale, and assessment of risk factors for postnatal depression. J Affect Disord 76:151–156

    Article  PubMed  CAS  Google Scholar 

  17. Bossuyt PM, Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig LM, Moher D, Rennie D, de Vet HCW, Lijmer JG (2003) The STARD statement for reporting studies of diagnostic accuracy: explanation and elaboration. Ann Intern Med 138:W1

    PubMed  Google Scholar 

  18. Boyce P, Stubbs J, Todd A (1993) The Edinburgh postnatal depression scale: validation for an Australian sample. Aust NZ J Psychiatry 27:472–476

    Article  CAS  Google Scholar 

  19. Carpiniello B, Pariante CM, Serri F, Costa G, Carta MG (1997) Validation of the Edinburgh postnatal depression scale in Italy. J Psychosom Obstet Gynaecol 18:280–285

    Article  PubMed  CAS  Google Scholar 

  20. Cochrane Diagnostic Test Accuracy Working Group (2008) http://srdta.cochrane.org/en/index.html. Accessed 19 February 2008

  21. Cooper PJ, Murray L, Hooper R, West A (1996) The development and validation of a predictive index for postpartum depression. Psychol Med 26:627–634

    PubMed  CAS  Google Scholar 

  22. Cox JL, Chapman G, Murray D, Jones P (1996) Validation of the Edinburgh postnatal depression scale (EPDS) in non-postnatal women. J Affect Disord 39:185–189

    Article  PubMed  CAS  Google Scholar 

  23. Cox JL, Holden JM, Sagovsky R (1987) Detection of postnatal depression. Development of the 10-item Edinburgh postnatal depression scale. Br J Psychiatry 150:782–786

    Article  PubMed  CAS  Google Scholar 

  24. Deeks JJ (2001) Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. BMJ 323:157–162

    Article  PubMed  CAS  Google Scholar 

  25. Eberhard-Gran M, Eskild A, Tambs K, Schei B, Opjordsmoen S (2001) The Edinburgh postnatal depression scale: validation in a Norwegian community sample. Nord J Psychiatry 55:113–117

    Article  PubMed  CAS  Google Scholar 

  26. Felice E, Saliba J, Grech V, Cox JL (2006) Validation of the Maltese version of the Edinburgh postnatal depression scale. Arch Womens Ment Health 9:75–80

    Article  PubMed  CAS  Google Scholar 

  27. Garcia-Esteve L, Ascaso C, Ojuel J, Navarro P (2003) Validation of the Edinburgh postnatal depression scale (EPDS) in Spanish mothers. J Affect Disord 75:71–76

    PubMed  Google Scholar 

  28. Ghubash R, Abou-Saleh MT, Daradkeh TK (1997) The validity of the Arabic Edinburgh postnatal depression scale. Soc Psychiatry Psychiatr Epidemiol 32:474–476

    PubMed  CAS  Google Scholar 

  29. Gilbody SM, House AO, Sheldon TA (2001) Routinely administered questionnaires for depression and anxiety: systematic review. BMJ 322:406–409

    Article  PubMed  CAS  Google Scholar 

  30. Guedeney N, Fermanian J (1998) Validation study of the French version of the Edinburgh postnatal depression scale: new results about use and psychometric properties. Eur Psychiatry 13:83–89

    Article  PubMed  CAS  Google Scholar 

  31. Harris B, Huckle P, Thomas R, Johnd S, Fung H (1989) The use of rating scales to identify postnatal depression. Br J Psychiatry 154:813–817

    Article  PubMed  CAS  Google Scholar 

  32. Health Technology Assessment Review The clinical validity, utility and cost effectiveness of methods for the identification of postnatal depression (PND) in primary care: an integrated evidence synthesis and value of information analysis. http://www.ncchta.org/project/1521.asp. (Accessed 19 February 2008)

  33. Holt WJ (1995) The detection of postnatal depression in general practice using the Edinburgh postnatal depression scale. N Z Med J 108:57–59

    PubMed  CAS  Google Scholar 

  34. Jadresic E, Araya R, Jara C (1995) Validation of the Edinburgh postnatal depression scale (EPDS) in Chilean postpartum women. J Psychosom Obstet Gynaecol 16:187–191

    Article  PubMed  CAS  Google Scholar 

  35. Jardri R, Pelta J, Maron M, Thomas P, Delion P, Codaccioni X, Goudemand M (2006) Predictive validation study of the Edinburgh postnatal depression scale in the first week after delivery and risk analysis for postnatal depression. J Affect Disord 93:169–176

    Article  PubMed  Google Scholar 

  36. Kitamura T, Shima S, Sugawara M, Toda MA (1994) Temporal variation of validity of self-rating questionnaires: repeated use of the general health questionnaire and Zung’s self-rating depression scale among women during antenatal and postnatal periods. Acta Psychiatr Scand 90:446–450

    Article  PubMed  CAS  Google Scholar 

  37. Lantz CA, Nebenzahl E (1996) Behavior and interpretation of the [kappa] statistic: Resolution of the two paradoxes. J Clin Epidemiol 49:431–434

    Article  PubMed  CAS  Google Scholar 

  38. Lawrie TA, Hofmeyr GJ, de Jager M, Berk M (1998) Validation of the Edinburgh postnatal depression scale on a cohort of South African women. S Afr Med J 88:1340–1344

    PubMed  CAS  Google Scholar 

  39. Lee DTS, Yip ASK, Chan SSM, Tsui MHY, Wong WS, Chung TKH (2003) Postdelivery screening for postpartum depression. Psychosom Med 65:357–361

    Article  PubMed  Google Scholar 

  40. Lee DTS, Yip ASK, Chiu HFK, Chung TKH (2000) Screening for postnatal depression using the double-test strategy. Psychosom Med 62:258–263

    PubMed  CAS  Google Scholar 

  41. Lee DTS, Yip ASK, Chiu HFK, Leung TYS, Chung TKH (2001) Screening for postnatal depression: are specific instruments mandatory? J Affect Disord 63:233–238

    Article  PubMed  CAS  Google Scholar 

  42. Leverton TJ, Elliott SA (2000) Is the EPDS a magic wand?: 1. A comparison of the Edinburgh postnatal depression scale and health visitor report as predictors of diagnosis on the Present State Examination. J Reprod Infant Psychol 18:279–296

    Article  Google Scholar 

  43. Matthey S, Barnett B, Kavanagh DJ, Howie P (2001) Validation of the Edinburgh postnatal depression scale for men, and comparison of item endorsement with their partners. J Affect Disord 64:175–184

    Article  PubMed  CAS  Google Scholar 

  44. Milgrom J, Ericksen J, Negri L, Gemmill AW (2005) Screening for postnatal depression in routine primary care: properties of the Edinburgh postnatal depression scale in an Australian sample. Aust NZ J Psychiatry 39:833–839

    Article  Google Scholar 

  45. Moher D, Schulz KF, Altman DG (2001) The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 357(9263):1191–1194

    Article  PubMed  CAS  Google Scholar 

  46. Murray L, Carothers AD (1990) The validation of the Edinburgh Post-natal Depression Scale on a community sample. Br J Psychiatry 157:288–290

    Article  PubMed  CAS  Google Scholar 

  47. Muzik M, Klier CM, Rosenblum KL, Holzinger A, Umek W, Katschnig H (2000) Are the commonly used self-report inventories suitable for screening postpartum depression and anxiety disorders? Acta Psychiatr Scand 102:71–73

    Article  PubMed  CAS  Google Scholar 

  48. Navarro P, Ascaso C, Garcia-Esteve L, Aguado J, Torres A, Martin-Santos R (2007) Postnatal psychiatric morbidity: a validation study of the GHQ–12 and the EPDS as screening tools. Gen Hosp Psychiatry 29:1–7

    Article  PubMed  Google Scholar 

  49. Ostrowski JA (2006) Accuracy of 3 diagnostic tests for anterior cruciate ligament tears. J Athl Train 41:120–121

    PubMed  Google Scholar 

  50. Pitanupong J, Liabsuetrakul T, Vittayanont A (2007) Validation of the Thai Edinburgh postnatal depression scale for screening postpartum depression. Psychiatry Res 149:253–259

    Article  PubMed  Google Scholar 

  51. Regmi S, Sligl W, Carter D, Grut W, Seear M (2002) A controlled study of postpartum depression among Nepalese women: validation of the Edinburgh Postpartum Depression Scale in Kathmandu. Tropical Med Int Health 7:378–382

    Article  Google Scholar 

  52. Rushidi WMWM, Amir A, Iran H, Mahmood NM (2004) Analysis of the psychometric properties of the Malay version of the Beck depression inventory II (BDI-II) among postpartum women in Kedah, North West of Peninsular Malaysia. Malays J Med Sci 11:19–25

    Google Scholar 

  53. Rushidi WMWM, Amir A, Mahmood NM (2003) Revalidation of the Malay version of the Edinburgh Postnatal Depresion Scale (EPDS) among Malay postpartum women attending the Bakar Bata Health Center in Alor Setar, Kedah, North West of Peninsular Malaysia. Malays J Med Sci 10:71–75

    Google Scholar 

  54. Sackett DL, Haynes RB (2002) Evidence base of clinical diagnosis: The architecture of diagnostic research. BMJ 324:539–541

    Article  PubMed  CAS  Google Scholar 

  55. Smidt N, Rutjes AWS, van der Windt D, Ostelo R, Bossuyt PM, Reitsma JB, Bouter LM, de Vet HCW (2006) The quality of diagnostic accuracy studies since the STARD statement: Has it improved? Neurology 67:792

    Article  PubMed  CAS  Google Scholar 

  56. Teng H-W, Hsu C-S, Shih S-M, Lu M-L, Pan J-J, Shen WW (2005) Screening postpartum depression with the Taiwanese version of the Edinburgh postnatal depression scale. Compr Psychiatry 46:261–265

    Article  PubMed  Google Scholar 

  57. Thompson WM, Harris B, Lazarus J, Richards C (1998) A comparison of the performance of rating scales used in the diagnosis of postnatal depression. Acta Psychiatr Scand 98:224–227

    Article  PubMed  CAS  Google Scholar 

  58. Uwakwe R (2003) Affective (depressive) morbidity in puerperal Nigerian women: validation of the Edinburgh postnatal depression scale. Acta Psychiatr Scand 107:251–259

    Article  PubMed  Google Scholar 

  59. Vittayanont A, Liabsuetrakul T, Pitanupong J (2006) Development of Postpartum Depression Screening Scale (PDSS): A Thai version for screening postpartum depression. J Med Assoc Thai 89(1):1–7

    PubMed  Google Scholar 

  60. Werrett J, Clifford C (2006) Validation of the Punjabi version of the Edinburgh postnatal depression scale (EPDS). Int J Nurs Stud 43:227–236

    Article  PubMed  CAS  Google Scholar 

  61. Whiffen VE (1988) Screening for postpartum depression: a methodological note. J Clin Psychol 44:367–371

    Article  PubMed  CAS  Google Scholar 

  62. Whiting P, Rutjes AW, Reitsma JB, Bossuyt PM, Kleijnen J (2003) The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Med Res Methodol 3:25

    Article  PubMed  Google Scholar 

  63. Whiting P, Rutjes AWS, Dinnes J, Reitsma JB, Bossuyt PMM, Kleijnen J (2005) A systematic review finds that diagnostic reviews fail to incorporate quality despite available tools. J Clin Epidemiol 58:1–12

    Article  PubMed  Google Scholar 

  64. Whiting PF, Weswood ME, Rutjes AW, Reitsma JB, Bossuyt PN, Kleijnen J (2006) Evaluation of QUADAS, a tool for the quality assessment of diagnostic accuracy studies. BMC Med Res Methodol 6

  65. Wickberg B, Hwang CP (1996) The Edinburgh postnatal depression scale: validation on a Swedish community sample. Acta Psychiatr Scand 94:181–184

    Article  PubMed  CAS  Google Scholar 

  66. Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weert HC (2007) Diagnostic accuracy of the mood module of the Patient Health Questionnaire: a systematic review. Gen Hosp Psychiatry 29:388–395

    Article  PubMed  Google Scholar 

  67. Yamashita H, Yoshida K, Nakano H, Tashiro N (2000) Postnatal depression in Japanese women: detecting the early onset of postnatal depression by closely monitoring the postpartum mood. J Affect Disord 58:145–154

    Article  PubMed  CAS  Google Scholar 

  68. Yoshida K, Marks MN, Kibe N, Kumar R, Nakano H, Tashiro N (1997) Postnatal depression in Japanese women who have given birth in England. J Affect Disord 43:69–77

    Article  PubMed  CAS  Google Scholar 

  69. Zelkowitz P, Milet TH (1995) Screening for post-partum depression in a community sample. Can J Psychiatry 40:80–87

    PubMed  CAS  Google Scholar 

Download references

Acknowledgments

We are grateful to the two anonymous referees for their comments to an earlier version of the paper.

Declaration of Intrest None.

Author information

Authors and Affiliations

  1. Dept. of Health Sciences, University of York, York, YO10 5DD, UK

    Rachel Mann MSc, Catherine E. Hewitt PhD &  Simon M. Gilbody Dphil, MRCPsych

Authors
  1. Rachel Mann MSc
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Catherine E. Hewitt PhD
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Simon M. Gilbody Dphil, MRCPsych
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mann, R., Hewitt, C.E. & Gilbody, S.M. Assessing the quality of diagnostic studies using psychometric instruments: applying QUADAS. Soc Psychiat Epidemiol 44, 300–307 (2009). https://doi.org/10.1007/s00127-008-0440-z

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00127-008-0440-z

Keywords

Search

Navigation