We discuss statistical methods to analyze data for reliability (agreement among or between raters) and validity. Such analysis are frequently required for medical care data. We also discuss the application of these statistical methods to a study of the reliability and validity of nominal group estimation procedures to identify cost-effective topics for quality assurance purposes.