Article Text

Download PDFPDF

First quality score for referral letters in gastroenterology—a validation study
  1. Sigrun Losada Eskeland1,
  2. Cathrine Brunborg2,
  3. Birgitte Seip3,
  4. Kristine Wiencke4,
  5. Øistein Hovde5,6,
  6. Tanja Owen7,
  7. Erik Skogestad8,
  8. Gert Huppertz-Hauss9,
  9. Fred-Arne Halvorsen10,
  10. Kjetil Garborg11,12,
  11. Lars Aabakken13,
  12. Thomas de Lange4,14
  1. 1Department of Medical Research, Vestre Viken Hospital Trust Bærum, Drammen, Norway
  2. 2Research Support Services, Oslo Centre for Biostatistics and Epidemiology, Oslo University Hospital, Oslo, Norway
  3. 3Department of Medicine, Vestfold Hospital Trust Tønsberg, Tønsberg, Norway
  4. 4Department of Medicine, Vestre Viken Hospital Trust Bærum, Drammen, Norway
  5. 5Department of Gastroenterology, Innlandet Hospital Trust Gjøvik, Gjøvik, Norway
  6. 6Institute of Clinical Medicine, University of Oslo, Oslo, Norway
  7. 7Department of Gastroenterology, Østfold Hospital Kalnes, Kalnes, Norway
  8. 8Department of Medicine, Innlandet Hospital Trust Lillehammer, Lillehammer, Norway
  9. 9Department of Gastroenterology, Telemark Hospital Trust Skien, Skien, Norway
  10. 10Department of Medicine, Vestre Viken Hospital Trust Drammen, Drammen, Norway
  11. 11Department of Transplantation Medicine, Oslo University Hospital, Oslo, Norway
  12. 12Department of Health Economics and Health Management, University of Oslo, Oslo, Norway
  13. 13Department of Gastroenterology, Oslo University Hospital, Oslo, Norway
  14. 14Cancer Registry of Norway, Oslo University Hospital, Oslo, Norway
  1. Correspondence to Sigrun Losada Eskeland; sigesk{at}


Objective To create and validate an objective and reliable score to assess referral quality in gastroenterology.

Design An observational multicentre study.

Setting and participants 25 gastroenterologists participated in selecting variables for a Thirty Point Score (TPS) for quality assessment of referrals to gastroenterology specialist healthcare for 9 common indications. From May to September 2014, 7 hospitals from the South-Eastern Norway Regional Health Authority participated in collecting and scoring 327 referrals to a gastroenterologist.

Main outcome measure Correlation between the TPS and a visual analogue scale (VAS) for referral quality.

Results The 327 referrals had an average TPS of 13.2 (range 1–25) and an average VAS of 4.7 (range 0.2–9.5). The reliability of the score was excellent, with an intra-rater intraclass correlation coefficient (ICC) of 0.87 and inter-rater ICC of 0.91. The overall correlation between the TPS and the VAS was moderate (r=0.42), and ranged from fair to substantial for the various indications. Mean agreement was good (ICC=0.47, 95% CI (0.34 to 0.57)), ranging from poor to good.

Conclusions The TPS is reliable, objective and shows good agreement with the subjective VAS. The score may be a useful tool for assessing referral quality in gastroenterology, particularly important when evaluating the effect of interventions to improve referral quality.


This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

View Full Text

Statistics from


  • Contributors SLE, TdL and LA designed the study. SLE, KW, ØH, BS, F-AH, TO, ES, KG and GH-H performed the data collection. SLE and CB performed the power and data analysis. SLE drafted the paper. All authors critically reviewed and improved it. SLE is the guarantor. All authors had access to all the data and take responsibility for the integrity of the data and the analysis.

  • Funding The PhD student salary was funded by the South-Eastern Norway Health Authority's research grant (grant agreement number 2008040) and the Norwegian Medical Association grant for quality improvement and patient safety (grant agreement number 14/1689). Researchers were independent of the funder.

  • Competing interests None declared.

  • Ethics approval The study was reported to and approved by the Data Protection Official for research. The Regional Ethics Committee considered the study outside its mandate, and its approval was not required. The presented data are anonymised and risk of identification is low.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement The full data set and statistical code can be made available from the corresponding author.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.