This study assesses the reliability of patient-reported information about care received by individual specialist physicians. A patient questionnaire that included core composites from the Consumer Assessment of Healthcare Providers and Systems Clinician & Group survey was administered to random samples of patients visiting 1315 physicians from 14 specialties in California during 2005-2006 (n = 68 406 respondents). The quality of specialist-patient interaction and organizational access composites achieved adequate physician-level reliability (alpha(MD) = 0.70) with 30 or fewer patients per specialist, but the care coordination and health promotion support composites were generally less reliable. Patients reporting consult-based relationships with specialists reported worse care experiences across measures (P < .001). The results indicate that reliable patient-reported information can be obtained about specialist physicians with patient sample size requirements comparable to primary care physicians. In order to promote equitable performance measurement in specialty care, future research should clarify the contribution of consult-based specialist-patient relationships to performance differences.