Public health programs rely on household-survey estimates of vaccination coverage as a basis of programmatic and policy decisions; however, the validity of estimates derived from household-retained vaccination cards and parental recall has not been thoroughly evaluated. Using data from a vaccination coverage survey conducted in the Western Pacific's Northern Mariana Islands, we compared results from household data sources to medical record sources for the same children. We calculated the percentage of children aged 1, 2, and 6 years who received all vaccines recommended by age 12 months, 24 months, and for school entry, respectively. Coverage estimates based on vaccination cards ranged from 14% to 30% in the three age groups compared to 78-91% for the same children based on medical records. When cards were supplemented by parental recall, estimates were 51-53%. Concordance, sensitivity, specificity, positive and negative predictive values, and kappa statistics generally indicated poor agreement between household and medical record sources. Household-retained vaccination cards and parental recall were insufficient sources of information for estimating vaccination coverage in this population. This study emphasizes the importance of identifying reliable sources of vaccination history information and reinforces the need for awareness of the potential limitations of vaccination coverage estimated from surveys that rely on household-retained cards and/or parental recall.