Objectives To quantify and analyse the quality of evidence that is presented in national guidelines.

Setting Levels of evidence used in all the current valid recommendations in the Scottish Intercollegiate Guideline Network (SIGN) guidelines were reviewed and statistically analysed.

Outcome measures The proportion of level D evidence used in each guideline and a statistical analysis.

Method Data were collected from published guidelines available online to the public. SIGN methodology entails a professional group selected by a national organisation to develop each of these guidelines. Statistical analysis of the relationship between the number of guideline recommendations and the quality of evidence used in its recommendations was performed.

Result The proportion of level D evidence increases with the number of recommendations made. This correlation is significant with Kendall's τ=0.22 (approximate 95% CI 0.008 to 0.45), p = 0.04; and Spearman ρ=0.22 (approximate 95% CI 0.02 to 0.57), p=0.04.

Conclusions Practice guidelines should be brief and based on scientific evidence. Paradoxically the longest guidelines have the highest proportion of recommendations based on the lowest level of evidence. Guideline developers should be more aware of the need for brevity and a stricter application of evidence-based principles could achieve this. The findings support calls for a review of how evidence is used and presented in guidelines.