Adapting and disseminating effective public health interventions in another country: towards a systematic approach

Abstract

Background: Public health interventions that have proven effective in one country, are often adapted and disseminated in other countries. However, the process by which effective interventions are chosen for adaptation and dissemination in another country is often not conducted systematically.

Methods: We propose a more systematic approach and describe the main steps that have to be taken in the assessment of the usefulness of effective interventions in another national context.

Results: The following steps are proposed. Step 0: Point out the most relevant areas in public health (this is a collaborative effort by policy makers, scientists and practitioners). Step 1: Identification of potentially effective interventions (through systematic literature searches). Step 2: Assessing the levels of evidence and grades of recommendation for adoption. Step 3: Can the results of the trials be generalized to the national situation? Step 4: Can the intervention be implemented in the national situation?

Conclusion: A more systematic approach to the adaptation and dissemination can be adopted. The basic steps described should be worked out in more detail before they can be used in practice.

The process of deciding whether or not to implement a promising new public health technology has been well-defined.^1–4 However, public health interventions that have proven effective in one country, are also often adapted and disseminated in other countries where they have not been examined. For example, in the area of psychosocial interventions a wide array of interventions has been adapted for use in other countries, such as the JOBS-programme (prevention of psychosocial problems among the unemployed);^5,6 the ‘Coping with Depression’ course (prevention and early treatment of depressive disorders;^7–9 the Australian Triple P programme (prevention and treatment of behaviour problems in children);^10,11 Communities that care (community programmes);¹² the PATHS curriculum for prevention of psychosocial problems at school;^13,14 the Nurse Home Visitation for the prevention of child abuse and neglect.¹⁵

Although the adaptation and dissemination of such interventions in another country are common practice, the process by which effective interventions are chosen for adaptation and dissemination is often not conducted systematically. Whether a new effective intervention is noticed depends very much on researchers and practitioners in a specific area and whether they keep up with the scientific literature in their field of study. It is entirely possible that important interventions are not ‘discovered’ in a country or only after a long time.

Furthermore, there are several methods and checklists to test the quality of research examining the effects of an intervention ^16,17 and to assess the innovative value.^1–4 However, there are no checklists to examine whether the contrast between experimental and control conditions examined in the international study, is also meaningful in a national context. For example, it is very well possible that a ‘care-as-usual’ condition in an American study is very different from the ‘care-as-usual’ in another country. In such a situation, the intervention should be examined in a new trial in the national situation, before it can be regarded as effective from a scientific point of view.

Also, interventions that have proven effective in one country may not always be implementable in another country. For example, the very strict ‘no use’ message in American prevention programmes for illegal drugs, does not fit in the public and professional opinions of several European countries. Or community programmes that have been examined in the communities of one country may not be useful in another country because the character of the communities differ strongly.

Because the resources for developing and examining public health interventions are limited in most smaller countries, the decision about which interventions are adapted to the national or local situation should be taken as efficiently and rationally as possible. It is, therefore, advisable to develop a more systematic approach. In this paper we will describe the main steps that have to be taken in the assessment of the usefulness of effective interventions in another national context.

In table 1, we have summarized the main steps of this process and important questions that have to be answered at each step. The decisions that can be taken after each step and the possible actions resulting from these decisions, are summarized in a decision tree (figure 1).

Step 0. Point out the most relevant areas in public health

Before interventions can be adopted in a country, it is advisable to define the most relevant public health areas in the country. The number of possible health care areas is very large, and it is not possible to examine the literature on all of these areas, or examine the research, and conduct pilot projects. It is possible, however, depending on the resources, to determine a number of health care areas on which the next four basic steps can be taken. In this step, scientists and policymakers should collaborate and combine knowledge about the prevalence and impact of health problems and political priorities. The impact of many health problems has been documented in most Western countries through various epidemiological studies, such as the Global Burden of Disease studies,¹⁸ and the replication of these studies for specific countries. In prioritizing health problems the impact of this problem on the number of disability adjusted life years in the country should be one of the major criteria.

Step 1. Identification of potentially effective interventions

The first step in the systematic assessment process is the identification of effective interventions which may be useful in another country. This step should be conducted through systematic literature studies. It is necessary to define the area which will be examined. For example, it is entirely feasible to examine whether effective interventions are available in the area of prevention of depression in adolescents, parent education, or prevention of work-related problems. Usually the selection of such areas will be conducted by financers of research and treatment or prevention services, such as governments or private funding agencies. The rules for conducting literature searches have been developed well, especially in recent years, through the work of the Cochrane Collaboration and other evidence based initiatives.¹⁹

Step 2. Assessing the levels of evidence and grades of recommendation for adoption

After identifying effect studies of interventions that may be useful in other countries, it is important to assess the strength of the evidence found in the trials examining the intervention. Several levels of evidence have been described elaborately in several studies, as well as several grades of recommendation for the adoption of an intervention.^1–4,20 Summarizing, these studies state that the evidence is stronger when the best designs have been used (well-conducted randomized controlled trials), when more studies have been conducted on the same intervention and when these trials have been integrated statistically into a meta-analysis. For the adoption of a new intervention it is also important that the cost-effectiveness has been examined in well-designed studies.²

Based on this evidence it is possible to develop grades of recommendation for the adoption of a new intervention. The recommendation to adopt a new intervention is strongest when the intervention has proven to be more effective than the existing interventions or when the costs of the new intervention are lower than the existing interventions.² For further information about the levels of evidence and the grades of recommendation for the adoption, the reader is referred to the many reviews and papers on this subject.^1–4,20

Step 3. Can the results of the trials be generalized to the national situation?

A fundamental problem in adapting an intervention to another national situation, is the generalizability of the trial results. Where medication is concerned, it is usually assumed that this is applicable in different cultures, as the physiology of human beings can be considered to be comparable. In public health interventions this is not automatically the case and it is entirely possible that there are fundamental differences. For example, it may well be the case that the ‘care-as-usual’ used for a control group in a US study differs strongly from the ‘care-as-usual’ in a European country. The results of such a study are not automatically useful in that European country. In that case a new trial has to be conducted in that country to examine whether this intervention is also effective in the other national context.

There is no clear-cut method for examining whether the results of international trials can be generalized to another national situation. This can only be substantiated by consulting national experts in the specific section of health care in which the intervention is to be implemented. Only this information can help in assessing whether the contrast of the international trial is also relevant to the national situation.

This step needs to be considered explicitly by groups of experts, as these are not issues for individual judgement. In these expert panels, both experts from practice should participate, as well as scientists and policymakers, as this is a discussion which touches on all three fields. It would be useful to develop a systematic description of the contrast examined in the trial, and the practice in the national situation. The contrast should be examined on three levels:

1. the professional delivering the service or conducting the intervention (expertise, training, resources, etc);

2. the target population (demographic characteristics, risk status);

3. the health care system (the financing system, the costs for the patient or care receiver, alternative interventions available etc).7

When the expert panel concludes that the results cannot be automatically generalized to the other national situation, a new national trial has to be conducted before broad dissemination of the intervention can be considered. When the results are generalizable, the next step can be taken.

Step 4. Can the intervention be implemented in the national situation?

When the intervention can be considered to be effective in the national situation, it has to be examined whether the intervention can be implemented. In order to reach this decision, the area of health care in which the intervention is to be implemented must be examined carefully. For example, it has to be clear that the professionals who will conduct the intervention are motivated to do so and have sufficient expertise, that sufficient resources for conducting the intervention are available or can be made available. It may be possible that comparable interventions are already in place on a large scale and that the professionals do not see the need to replace these.

Again, this is not an issue for individual judgement, and this step should also be considered explicitly by a group of experts from practice, science and policymakers. The question whether the intervention is implementable should also be considered on the same three levels as described in step 3: the level of the professional delivering the new service (do they sufficient knowledge and skills, time and resources to deliver the intervention?); the individual level (will the intervention be acceptable for the target population in this format?); and the health care system (does the intervention fit into the health care system in terms of financing, alternative interventions, resources, priorities?).

When all signs seem to indicate that the intervention can be implemented, it is advisable to translate the manuals and materials, and to conduct some pilot projects in two or three regions. This is the only way to discover practical problems and further adaptation and improvement to the intervention. If these pilot projects are successful, then there is no further barrier for broad-scale implementation of the intervention in the national situation.

If the conclusion is that the intervention cannot be implemented in the national situation, it may be useful to examine if the results of the trials can be used in other ways, for example by adapting the contents of existing interventions.

Discussion

In this paper we argued that the decision whether an intervention that has proven effective in one country should be implemented in another country, has to be rationalised. We proposed four basic steps that have to be taken before such an intervention can be implemented in another country. Of course we only made a first draft of a possible decision process. We described basic elements, but this approach should be worked out in detail before it can be used in practice.

Although our approach seems to be more rational than the ad hoc methods that are now common practice, several problems remain unsolved. First, this approach is based on the availability of well-designed, randomized trials. In several important public health areas, such as mass media campaigns, it is virtually impossible to conduct randomized trials. Such interventions may be very important and useful, but the effects cannot be examined without major risks of bias. In this case the basic steps of our approach, especially step 2, are problematic. Second, when the basic steps are followed and the conclusion is that an intervention can be implemented on a wide scale, it is never certain whether the translation and the adaptations to the national context have reduced the effectiveness of the intervention. Therefore, one may argue that it is always advisable to conduct a new trial examining the effects of the intervention in the national context.

A major challenge for scientist and policymakers is to define the areas of health care they consider to be the most important. This is a complex enterprise, demanding policymakers and scientist to work together closely. National Health Boards could play a major role in this and several have developed systematic strategies to prioritize health subjects and the development and adoption of new interventions, as in the United Kingdom and in The Netherlands. But this cannot be a one-off action. Public health care is undergoing rapid development, and effective, innovative interventions are appearing regularly. National governments should reserve resources for defining the areas they want to have screened for effective interventions, and for repeating this action regularly, for example every year. This is the only way to decide rationally which interventions should be implemented in order to reduce the health problems that are considered most important.

References