Article Text
PDF
Abstract
Objectives We sought the perspectives of lead public health officials working to improve health equity in the USA regarding the drivers of scientific evidence use, the supply of scientific evidence and the gap between their evidentiary needs and the available scientific evidence.
Design We conducted 25 semistructured qualitative interviews (April 2017 to June 2017) with lead public health officials and their designees. All interviews were transcribed and thematically analysed.
Setting Public health departments from all geographical regions in the USA.
Participants Participants included lead public health officials (20) and their designees (5) from public health departments that were either accredited or part of the Big Cities Health Coalition.
Results Many respondents were using scientific evidence in the context of grant writing. Professional organisations and government agencies, rather than specific researchers or research journals, were the primary sources of scientific evidence. Respondents wanted to see more locally tailored cost-effectiveness research and often desired to participate in the planning phase of research projects. In addition to the scientific content recommendations, respondents felt the usefulness of scientific evidence could be improved by simplifying it and framing it for diverse audiences including elected officials and community stakeholders.
Conclusions Respondents are eager to use scientific evidence but also need to have it designed and packaged in ways that meet their needs.
- qualitative research
- public health
- health policy
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This is one of the few studies to explore the evidentiary needs of lead public health officials in the USA working to improve health equity.
We have respondents from health departments in all geographical regions of the USA.
We only include respondents from either accredited health departments or departments in the Big Cities Health Coalition.
In most cases, we only interview one individual from each department so their views may not reflect the views of other individuals in the department.
Introduction
Achieving health equity, attainment of the highest level of health possible for all people, continues to be a challenge in the USA.1 Adult men and women without a high-school diploma can expect to live an average of 7 and 5 years shorter than their counterparts with 16 or more years of formal education, respectively.2 Health disparities continue to persist between white and some racial/ethnic minority populations.3 The Commission on Social Determinants of Health highlights strengthening the evidence base for health inequity, the social determinants of health and what works to improve them as key to making progress on health equity.4
Empirical work exploring the perspectives of policy advisors addressing health equity regarding the current evidence base and their evidentiary needs reveals that they find the bulk of scientific evidence irrelevant to their policy-making efforts.5 The poor fit between the evidence and the development of policies to promote health equity has been attributed to two central issues: (1) naiveté, on the part of researchers, about the context of policy-making and (2) scientific evidence that is not easily translatable to the real world. Specifically with regards to the political context, policy advisors argue that researchers seldom appreciate the time constraints they work under, their need to take action expeditiously with or without scientific evidence and the need to craft politically feasible policy solutions. As to the content of scientific evidence, policy advisors addressing health equity would like to see more evaluative rather than descriptive research, more discussion of the mechanisms underlying scientific results, an illustration of the health equity dimension of the results and more information on the cost of proposed policy interventions.5
Most of what is known about the gap between the evidentiary needs of policy-makers addressing health equity and the available scientific evidence comes from policy-makers outside of the USA.5 To our knowledge, the one study to address this topic with policy-makers had seven respondents and only one of them was from the USA.5 We sought to understand the extent of this gap among policy-makers addressing health equity in the USA. To explore these issues, researchers from the UCLA Fielding School of Public Health Center for Health Advancement, with support from the de Beaumont Foundation, conducted semistructured qualitative interviews with lead public health officials nationwide.
Methods
We purposefully sampled health departments that had some likelihood of engaging in work to improve health equity.6 Our sample was drawn from two sampling frames: all 28 public health departments participating in the Big Cities Health Coalition (BCHC) (http://www.bigcitieshealth.org/) and all accredited public health departments. The BCHC is a forum for America’s largest metropolitan area health departments to exchange strategies to improve public health, with a mission to advance equity and health for present and future generations. In order to be eligible for participation in the BCHC health departments must be locally controlled and have a population of at least 400 000. According to the 2016 National Profile of Health Departments, relative to smaller health departments (population <500 000), larger health departments are more likely to have community health workers and epidemiologist on staff, positions that can potentially support efforts to address health equity.7 Accredited departments were included because they must demonstrate engagement with the community to identify and address health problems as a condition of accreditation. Community engagement has been identified as an effective strategy to improve health equity.8 We first selected accredited public health departments for variability on criteria that we thought may influence their approach to work on health equity, including geographical region (west, midwest, southwest, northeast and southeast) and organisational structure (state vs local). Because the BCHC departments were large, we limited inclusion of local accredited public health departments to the smallest public health departments (catchment size <53 000). Ultimately, we selected 32 of the 189 accredited public health departments for inclusion in the sample. This study was granted exempt status by the University of California Los Angeles Institutional Review Board.
Researchers sent email invitations to participate in the study to 60 public health departments. Invitations sent to the 28 BCHC members also included an email endorsing the study from BCHC leaders. A second round of emails was sent, followed by one round of phone calls. A total of 25 public health directors (20) and individuals designated by directors (5), were interviewed between April and June of 2017 (response rate of 42%). An experienced research assistant obtained verbal consent and conducted phone interviews.
A phenomenological interview framework was used to conduct all interviews. This approach uses open-ended questions to probe participants to reconstruct their experiences and reflect on their meaning.9 The interview domains were largely based on domains used in extant literature on the topic of evidentiary needs in the context of working to improve health equity, among policy advisors. Consistent with the previous study of this topic among policy advisors, these domains addressed scientific evidence generally and included drivers of scientific evidence use, sources of scientific evidence and ways to make scientific evidence more useful (supplementary file).5 In addition to asking about preferences for scientific evidence generally, we also inquired specifically about perceptions of economic modelling/analysis. The interviews lasted between 30 and 75 min and were tape recorded and transcribed.
Supplementary file 1
Two researchers (KN and JR) independently coded transcripts using a structural coding process, which involved applying a content-based phrase representing a particular topic to a segment of data relating to a particular research question.10 All coding was deductive. The researchers initially compared coding from two transcripts and discussed their findings. The coding across the two researchers was highly consistent. In the couple of instances when coding discrepancies emerged, the codebook was reviewed to settle the dispute. If review of the codebook did not resolve the discrepancy, new codes were created or old codes were redefined. The researchers then independently coded the remainder of the transcripts and generated candidate themes. Another meeting was held to compare candidate themes across researchers. Only candidate themes identified by both researchers were carried forward. The themes identified were highly consistent across researchers.
Patient involvement
Patients were not involved.
Results
The sample included 22 local health departments and 3 state health departments. Sixteen departments were members of the BCHC and 9 non-BCHC departments came from the accreditation list. Eight of the 16 BCHC departments in the sample were also accredited. Health departments from all US Census geographical regions were included in the sample (west (8), midwest (6), northeast (4), southeast (4) and southwest (3)). The population served by participating departments ranged from 23 000 to just over 2 million with a median of 682 545.
Seven major themes emerged from the analysis: (1) funders are key drivers of scientific evidence use; (2)The Centers for Disease Control and Prevention (CDC) and national organisations are important suppliers of scientific evidence; (3) information on intervention cost effectiveness is key to addressing health equity; (4) information on intervention effectiveness at the local level is essential for compelling stakeholders to act; (5) framing scientific evidence to resonate with different audiences is important for increasing the impact of the evidence; (6) use of simple terminology to describe scientific evidence is essential for engaging community stakeholders and (7) summaries of science and systematic reviews are highly valuable (table 1).
Theme frequency table
Funders are key drivers of scientific evidence use
The explicit requirements by funders to use scientifically supported interventions emerged as one of the key drivers of the use of scientific evidence. Many of these public health departments relied heavily on grants from the government and foundations to support their health equity work.
Most of the funding opportunities we have requires the use of interventions that are evidence-based… we usually look to CDC or other sources for those.
Additionally, funders are able to influence what type of scientific evidence is being used by providing grants for the implementation of specific types of interventions.
We’re not looking at the evidence and saying we have all this money that we need to channel into a new STD prevention program because we see that STDs are on the rise. We don’t really have that nimble capacity because our funding is so tied to grants.
In some cases, the activities undertaken by health departments may be more reflective of the current priorities of funders than the areas of greatest need for the population being served.
The CDC and national organisations are important suppliers of scientific evidence
Departments typically relied on the CDC and other national organisations for scientific evidence.
I’m not sure that academia knows how to get their information in front of public health departments. Using existing structures, such as NACCHO or ASTHO, or the Big City Health Coalition to disseminate information is more effective than creating new ones.
Rather than looking to the work of specific individuals for scientific evidence these departments looked to trusted sources for scientific evidence when they were not generating their own data. Notably, respondents infrequently mentioned abstracting scientific evidence directly from peer-reviewed journals.
Information on intervention cost effectiveness is key to addressing health equity
Nearly half of the respondents were interested in having more information on the cost effectiveness of interventions and believed this would facilitate progress on health equity.
If legislators are responsible for balancing the budget and making fiscal decisions that are wise, and if they’re needing to make decisions that are in the best interests of their constituents, their communities, as well as the state, I think they need to be able to see a complete package of information for an intervention. We can provide them with information on the science of an intervention but at the end of the day, that economic piece is so critical.
Having information on cost was seen as particularly salient among departments facing budget shortfalls. In addition to having information on cost, departments wanted this information on a local scale and wanted to know how cost and benefits would be allocated across sectors.
I would say if economic modeling is being used, the one wish I would have is it be done not from the societal perspective but from a more narrative perspective of either the government or the payers, because decisions are made by those players based on their own economic interests, and if you do it from a societal perspective, it is not as persuasive.
While having information on the cost and cost effectiveness of interventions was seen as extremely valuable to most departments, there were substantial barriers to obtaining and using this information. The most prevalent barrier was the lack of technological expertise to generate and use cost-effectiveness information. The following quote outlines some of the challenges faced with using information derived from economic modelling.
Not just lack of skills, but probably lack of experience or knowledge, on my part included. What does this information really mean? How do I best phrase it for someone who maybe isn’t from a public health background, like a board member, or a member of one of my work groups that’s in law enforcement, or in a non-profit community organization?
Departments were very interested in having an entity that could reliably provide intervention cost information to them.
I think what would be needed is an organization that could focus on economics and cost benefit that could be a resource for others departments to use.
Information on intervention effectiveness at the local level is essential for compelling stakeholders to act
While some departments were interested in national-level evaluations of interventions most were looking for smaller-scale evaluations.
What really speaks to the communities that we work with and speaks to the political leaders is having information that is available on a more granular, community-level scale.
Additionally, departments wanted evaluative research that provided enough contextual information about the intervention community to allow for tailoring of the intervention to local needs.
We have both an urban center but also very rural areas and things that work in one community may not be feasible to execute in areas where it is quite rural. So understanding the key aspects of a particular intervention and what parts can be either changed or not utilized, and with minimal effect on the impact would be helpful.
One proposal presented for increasing the match between departmental evidence needs, and the scientific evidence produced was collaboration between departments and researchers prior to generation of research rather than presentation of research to departments after it is produced.
Having public health practitioners who are involved in the state and local level involved in the design and the implementation of the research studies will help the data that are generated from those studies to be more easily packaged and disseminated for implementation and practice.
Framing scientific evidence to resonate with different audiences is important for increasing the impact of the evidence
Departments felt that focusing on the framing of scientific evidence for diverse audiences was an important way for scientist to strengthen the presentation of scientific evidence.
I think that the information and data are there, and it’s helpful, but sometimes, translating it to talking points that are appropriate for different audiences is lacking, or building in what would be important for what audiences, or considerations depending on what kind of state we live in.
Use of simple terminology to describe scientific evidence is essential for engaging community stakeholders
While delivering scientific evidence in a way that resonated with public health and elected officials was seen as key; respondents also spoke to the importance of delivering scientific evidence in a way that resonated with community stakeholders.
I think it’s not just for me, but it’s also for the community, and the community, I’m using in the broader sense, thinking about residents as well as stakeholders, organizations, professionals, profit and non-profit. It can’t be too scientific in its explanation, it has to have both what I would call a heart and a brain statements in it, what is it that touches and inspires people, but also, what is it that explains things in a way that’s simple enough for everyone to understand?
Summaries of science and systematic reviews are highly valuable
Several respondents articulated their preference for scientific summaries and systematic reviews as a means of consuming scientific information effectively and efficiently.
Syntheses would be fantastic. People doing critical systematic reviews on the emerging topics in public health would be a huge value…
Scientific summaries that graded the available evidence base for a given intervention and that made concrete recommendations regarding intervention implementation were viewed as particularly valuable.
Discussion
We interviewed 25 lead public health officials working to improve health equity in the USA and their designees regarding drivers of scientific evidence use, their current supply of scientific evidence and ways to improve the uptake of scientific evidence. There were novel findings in this work that were not present in prior studies of this topic among policy advisors or researchers primarily in the UK, such as the importance of funding agencies for facilitating the use of scientific evidence.5 11 12 Particularly, we found that the CDC was both a key driver of scientific evidence use, through their grant-making efforts and a source of scientific evidence. This represents an important divide between how scientist typically disseminate evidence and how public health departments retrieve evidence, which has implications for the incorporation of scientific evidence into policy.
One finding that was consistent with the results of studies with respondents primarily from the UK was the desire for more evidence on the cost effectiveness of interventions. Cost has come up as an important driver of policy-making decisions around health equity in multiple studies based in the UK5 11, and this study reiterates these findings while also underlining the fact that data on costs and cost effectiveness of interventions specific to local jurisdictions, conducted from the perspective of payers rather than society, and that is easily interpretable is ideal. Given the dependence of many departments on grants from government to perform health equity work and projected budget cuts from the federal government, cost-effectiveness information may become even more valuable to policy-makers. Tailoring of evidence to local jurisdictions has also been mentioned as important in studies based out of the UK.5 11 One way for public health departments to obtain more locally tailored evidence that was suggested by one of the respondents was having more participation of public health departments in the planning phase of the studies. The research approach being advocated for seemed to be more similar to community-based participatory research, with public health departments functioning like community partners.13 Additionally, it was thought that the uptake of scientific evidence can be enhanced by framing the evidence for diverse audiences and simplifying descriptions of the science. Developing stories of impact that are fully consistent with scientific evidence—‘heart and brain statements’ as one respondent put it—would be a useful form of academic research. The emphasis on the necessity of framing scientific evidence for policy-makers and elected officials is not new and has been mentioned among policy advisors and researchers in the UK, but the emphasis on framing scientific evidence to resonate with community stakeholders has not been previously mentioned.5 This sentiment may reflect the pivot of departments focused on improving health equity towards more inclusive approaches to policy-making that relies heavily on community input. Finally, some respondents favoured summaries and systematic reviews as a means of consuming scientific evidence. This sentiment is not novel and has been expressed during studies of this topic among policy advisors and researchers primarily based in the UK.5 11
The findings of this study must be interpreted in the context of certain limitations. Our study population included a small number of mostly urban public health departments. Fifty per cent of our respondents served populations greater than 500 000. According to the 2016 National Profile of Health Departments, only 6% of local health departments serve catchment sizes that large; however, such large public health departments serve more than 50% of the population in the USA.7 Consequently, the findings of this study may not be attributable to most public health departments. The individuals interviewed provided their own views, and these views may differ from others even within their own departments. We also do not know from this current research project what objective progress departments have made in their efforts to promote health equity.
Respondents are eager to use scientific evidence, but also need to have it designed, executed and packaged in ways that meets their needs. Policy-makers want to know how it would work in their own area, for whom, and at what cost.
Footnotes
Contributors KDCN, FJZ, JF, ST, NR, BC: conceived this project. JR, NR, KDCN: carried out the qualitative interviews. All authors performed data analysis and contributed to the final manuscript.
Funding This work was supported by the de Beaumont Foundation, grant number 20164698.
Competing interests None declared.
Patient consent Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data are available.