This report describes the approach the National Kidney Foundation-Dialysis Outcomes Quality Initiative (NKF-DOQI) used to assess the strength of published evidence pertinent to individual NKF-DOQI Clinical Practice Guidelines, as well as the relationship between that approach and methods used by the US Preventive Services Task Force, the Cochrane Collaboration, and the Agency for Health Care Policy and Research to rate the quality and/or strength of evidence. We also present the results of an analysis of the strength of evidence underlying the NKF-DOQI Guidelines showing that one cannot infer the quality of evidence reported in a study (rated either on a 0-to-1 scale or categorically as excellent, very good, good, fair, or poor) simply by knowing the type of study design used (randomized trial, nonrandomized trial, natural experiment, cohort study, cross-sectional study, case-control study, case report). Issues related to assessment of the strength of evidence underlying a practice guideline opposed to that reported in an individual study are highlighted.