With the growing importance of health in the global agenda comes the responsibility to develop a scientific foundation of metrics and evaluation. The scope of this emerging field can be viewed in terms of key topics, including health outcomes, other social outcomes related to health systems, health services, resource inputs, evaluations of programmes and systems, and analyses to support policy choice. It can also be defined in terms of key activities that are needed to strengthen the scientific basis of the field: development of new methods, instruments, software, and hardware; setting global norms and standards for data collection; increasing the availability of high-quality primary data; systematic analysis and synthesis of existing datasets; strengthening national capacity to obtain, analyse, and use data; and reporting and disseminating results. We explore in depth topics with major scientific challenges and institutional and cultural barriers that are slowing the development of the field. Cutting across the various topical areas and disciplinary approaches to these problems are some common scientific issues, including limited comparability of measurement, uncorrected known biases in data, no standard approach to missing data, unrealistic uncertainty estimates, and the use of disease models that have not been properly validated. Only through concerted action will it be possible to assure the production, reproduction, and use of knowledge that is crucial to the advancement of global health.