Article Text

Download PDFPDF

Effectiveness of a multicentre nasopharyngeal carcinoma awareness programme in Indonesia
  1. Renske Fles1,
  2. Sagung R Indrasari2,
  3. Camelia Herdini2,
  4. Santi Martini3,
  5. Atoillah Isfandiari3,
  6. Achmad C Romdhoni4,
  7. Marlinda Adham5,
  8. Ika D Mayangsari5,
  9. Erik van Werkhoven6,
  10. Maarten A Wildeman7,
  11. Bambang Hariwiyanto2,
  12. Bambang Hermani5,
  13. Widodo A Kentjono4,
  14. Sofia M Haryana8,
  15. Marjanka K Schmidt9,10,
  16. I Bing Tan1,2,11
  1. 1Department of Head and Neck Surgery and Oncology, The Netherlands Cancer Institute, Amsterdam, The Netherlands
  2. 2Department of Otorhinolaryngology, Dr. Sardjito General Hospital, Faculty of Medicine Gadjah Mada University, Yogyakarta, Indonesia
  3. 3Department of Public Health, Airlangga University, Surabaya, Indonesia
  4. 4Department of Otorhinolaryngology, Dr. Soetomo Hospital, Faculty of Medicine Airlangga University, Surabaya, Indonesia
  5. 5Department of Otorhinolaryngology, Dr. Cipto Mangunkusumo Hospital, Faculty of Medicine University of Indonesia, Jakarta, Indonesia
  6. 6Department of Biometrics, The Netherlands Cancer Institute, Amsterdam, The Netherlands
  7. 7Department of Otorhinolaryngology, Academic Medical Centre, Amsterdam, The Netherlands
  8. 8Department of Histology, Cell and Tumour Biology, Faculty of Medicine, Gadjah Mada University, Yogyakarta, Indonesia
  9. 9Division of Psychosocial Research and Epidemiology, Netherlands Cancer Institute, Amsterdam, The Netherlands
  10. 10Division of Molecular Pathology, Netherlands Cancer Institute, Amsterdam, The Netherlands
  11. 11Department of Oral and Maxillofacial Surgery, Academic medical Centre, Amsterdam, The Netherlands
  1. Correspondence to Dr Marjanka K Schmidt;{at}


Objective To evaluate the effectiveness of a nasopharyngeal carcinoma (NPC) awareness programme on the short-term and long-term improvement of knowledge and referral of patients with NPC by primary healthcare centres (PHCCs) staff in Indonesia.

Design The NPC awareness programme consisted of 12 symposia including a Train-The-Trainer component, containing lectures about early symptoms and risk factors of NPC, practical examination and the referral system for NPC suspects. Before and after training participants completed a questionnaire. The Indonesian Doctors Association accredited all activities.

Participants 1 representative general practitioner (GP) from each PHCC attended an NPC awareness symposium. On the basis of the Train-The-Trainer principle, GPs received training material and were obligated to train their colleagues in the PHCC.

Results 703 GPs attended the symposia and trained 1349 staff members: 314 other GPs, 685 nurses and 350 midwives. After the training, respondents’ average score regarding the knowledge of NPC symptoms increased from 47 points (of the 100) to 74 points (p<0.001); this increase was similar between symposium and Train-The-Trainer component (p=0.88). At 1½ years after the training, this knowledge remained significantly increased at 59 points (p<0.001).

Conclusions The initial results of this NPC awareness programme indicate that the programme effectively increases NPC knowledge in the short and long term and therefore should be continued. Effects of the improved knowledge on the stage at diagnoses of the patients with NPC will still need to be scrutinised. This awareness programme can serve as a blueprint for other cancer types in Indonesia and for other developing countries.

  • awareness
  • nasopharyngeal carcinoma

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

Statistics from

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

Strengths and limitations of this study

  • This is the first study evaluating a multicentre education programme dealing with one of the most frequent cancers affecting males in Indonesia.

  • The study demonstrates improved knowledge of nasopharyngeal carcinoma among healthcare workers in the primary healthcare centres.

  • This study also demonstrates that in the long term the effect of the education programme is still present.

  • No individually paired scores were available since all questionnaires were completed anonymously.

  • It was not possible to evaluate the effects on stage shift due to earlier referral.


Nasopharyngeal carcinoma (NPC) is uncommon in most parts of the world; however, it is a major burden in Indonesia with 15 000 new cases a year.1 This number might even be an underestimation due to inadequate cancer registries. NPC is very sensitive to (chemo-)radiotherapy, resulting in a 2 and 3 year survival of 84% and 78%, respectively,2 provided that patients present themselves with early stage cancer. However, early symptoms of NPC are often minor and non-specific, making it difficult to recognise NPC suspects for timely referral. For example, 88–97% of patients with NPC in two major Indonesian hospitals (Dr. Cipto Mangunkusumo Hospital, Jakarta; and Dr. Sardjito Hospital, Yogyakarta) developed advanced NPC before presenting at the hospital.3 ,4 In line with this finding, an earlier observational study in Indonesia, including patients treated with curative intent, showed a complete response of 29% directly after treatment.4

The symptoms of NPC can be subdivided into four different categories, namely: (1) tumour mass in the nasopharynx causing blood-tinged secretion, nasal obstruction and sometimes epistaxis; (2) dysfunction of the Eustachian tube inducing fullness, hearing loss and tinnitus; (3) skull base erosion and palsy of the third to sixth cranial nerves provoking headache, diplopia, facial pain and numbness including eye symptoms such as strabismus and lagophthalmos3 and (4) neck mass.5 Well-established risk factors for NPC are the Epstein-Barr virus,6 family history of NPC (4–10-fold increase), ethnicity and gender.7–10 Environmental and lifestyle risk factors are the consumption of salted fish,7 ,10–12 usage of herbal medicine,10 ,13 wood dust exposure and other occupational exposures such as fume, smoke and chemicals.7 ,14 ,15 Multiple studies reported also on the increased risk of NPC caused by smoking.8 ,16–18

Midwives and nurses play an important supportive role in the work of the general practitioners (GPs) in the primary healthcare centres (PHCCs). In Indonesia, there are on average only 13 GPs available per 100 000 inhabitants. Hence, the PHCCs are often without an attending GP, and the nurses and the midwives then take over their role as medical doctors without having the right educational degree or knowledge to do so.19–21 We showed before that the knowledge on NPC of the GPs working in a PHCC is limited and that creating more awareness is of great importance to minimise doctors’ delay.22–26 Therefore, the NPC awareness programme started in 2009 in Jakarta, Yogyakarta and Surabaya. The first results showed an increase in the short-term knowledge of the GPs working in the PHCC using different teaching methods and evoked continuation and expansion of the NPC awareness programme to other cities.27

This study investigated the effects of the awareness symposia conducted at different locations in Indonesia, and of a Train-The-Trainer programme for all medical healthcare workers in the PHCC. The aim of this study was to evaluate the effectiveness of this combined NPC awareness programme on the short-term and long-term improvement of knowledge and referral of patients with NPC by PHCC staff.


Study population

The study population comprised healthcare workers working in the PHCCs in the provinces of Yogyakarta, Jakarta and East Java.

NPC awareness programme

The NPC awareness programme consisted of two activities: GPs attended an NPC awareness symposium in their own region, and then organised a Train-The-Trainer programme. Participants of the symposium received course materials on a CD, which contained presentations about the symptoms, practical examination, referral, diagnostics and treatment of NPC. They also received posters, flyers and a booklet with information on NPC. Participants were obliged to train their colleagues in their own PHCC. The direct effects and, after a minimum interval of 1½ years, the long-term effect of these two training methods were measured using structured questionnaires. The awareness programme was held between January 2010 and December 2011 in different cities in the provinces Jakarta, Yogyakarta and East Java; the long-term effect was measured between November 2012 and February 2013 in the province of Yogyakarta (figure 1).

Figure 1

Overview of the nasopharyngeal carcinoma (NPC) awareness programme. The Train-The-Trainer programme followed the symposium. One and a half year after this programmes was finished, primary healthcare centres (PHCCs) were visited again to measure the long-term effect.

NPC awareness symposium

The NPC awareness symposium was a 1-day training session, which consisted of lectures focusing on the risk factors, symptoms and incidence of NPC; the referral system concerning NPC suspects; testimonials of patients; and demonstration videos. In addition, participants took part in a practical examination training of the head and neck areas in small groups. The Indonesia Doctors Association (IDI) accredited the event. The symposia were organised by the departments of otorhinolaryngology of the participating centres, in collaboration with the Netherlands Cancer Institute, Amsterdam. Additionally, in Yogyakarta and east Java, the departments of public health of, respectively, the Gadjah Mada University, Yogyakarta and the Airlangga University, Surabaya, were involved.

NPC Train-The-Trainer programme

In the provinces Yogyakarta and East Java, one GP per PHCC joined the NPC awareness symposium. After this training, the GP had the obligation to train his colleagues in the PHCC using the materials handed out during the symposium. This training in the PHCC took place within 1 month after the symposium and was supervised by one of the NPC awareness team members or a staff member from the local health department. In addition to GPs, this Train-The-Trainer programme was attended by nurses, midwives, dentists and other public healthcare workers. We removed 57 questionnaires from the data analyses, which were filled in by non-clinical staff. IDI also accredited this Train-The-Trainer programme. Credits and the certificate were awarded once the questionnaires of the participants were returned to a member of the NPC awareness team.


Data were collected using a written, structured questionnaire before and after the symposia or Train-The-Trainer programme as described previously.22 In short, the questionnaire consisted of questions assessing general information about the participants and their work environment, for example, years of work experience; general knowledge on NPC such as early symptoms and risk factors; and experience of dealing in daily practice with NPC suspects in their PHCC. In the questionnaire, participants were asked to list all NPC symptoms and risk factors, up to a maximum of seven. One and a half year after the NPC awareness programme, all PHCCs in the province Yogyakarta were revisited, and all present medical workers were asked to complete the same questionnaire. Questions about their attendance of previous NPC symposia or Train-The-Trainer programme, and if this training had resulted in diagnosing more NPC suspects in their PHCC, had been added. All questionnaires were completed anonymously.

Participation and response

The Public Health Offices (Dinas Kesehatan) of the provinces and districts approved the NPC awareness programme. Moreover, the Indonesian doctors associated approved and accredited the activities. By involving the public health offices at province and district levels, all participants received an official invitation to attend the awareness symposium, and participation was obligatory. In addition, they were obligated to train their colleagues in their own PHCC. A member from the NPC awareness team or a representative from the public health office supervised this training. For the follow-up questionnaires to measure the long-term effect, an NPC awareness team member visited all PHCCs in the province Yogyakarta that had joined the awareness programme; questionnaires were completed by the health staff present at the time of the visit(s).

Statistical analysis

Respondents’ performance on symptoms and risk factors was assessed by analysis of the fraction of correct answers, weighted by the total number of answers, with a generalised linear model with binomial error distribution and identity link function. This model has the advantage that the estimates may be interpreted directly in terms of probabilities, whereas with a logistic model they would yield ORs. For convenience of representation and discussion, the probabilities were rescaled to percentage points (ie, multiplied by 100). Answers to single questions were analysed with logistic regression.

Training was grouped into pretraining, postsymposium and post-Train-The-Trainer programme. Profession (GP, midwife, nurse or other) and years of work experience (<3, 3–6, 6–9 or >9 years) were entered as covariates. Since many respondents (n=1493) did not fill in the question about their work experience, a separate category was created for missing values. If any of the other variables were missing, the observation was removed from the analysis. A separate model with an interaction term was used to evaluate the Train-The-Trainer programme for different professions. Analyses were done using the R-package V.3.1.1; all presented p values are two sided.


Participants of the symposia and Train-The-Trainer programme

In total, 5309 questionnaires were completed: at the symposia, 709 pretraining and 715 postsymposium, at the Train-The-Trainer programme 1577 pretraining and 1563 post-Train-The-Trainer programme, and 735 participants filled out the questionnaire 1½ years later (table 1). At the symposia, a representative GP from every PHCC was invited. The Train-The-Trainer programme was open to all medical healthcare workers in the PHCC of the representative; 21% of the participants were GPs. More than 50% of the participants, independent of profession or type of training, had <6 years’ work experience. There were three or less GPs working in the PHCC of 72% of the participants. In order to obtain the long-term effect of the awareness programme, all PHCCs in Yogyakarta were revisited and all medical healthcare workers were asked to complete the same questionnaire again. Participants stated if they had participated in one of the NPC trainings previously; this was indicated by 180 (24%) participants (table 1).

Table 1

Overview of the different trainings within the NPC awareness programme

General NPC questions

Before any kind of training, participants were asked if they thought NPC was a serious problem; 94% of the participants agreed or strongly agreed with this statement; and 97% of the participants stated that patients had a better chance to survive when treated at an earlier stage. After the symposium and Train-The-Trainer programme, almost everybody agreed with those statements, respectively, 98% and 99% (p values <0.001); at the 1½ years assessment, 96% of the participants stated that NPC was a serious problem (p=0.02), and 96% agreed that patients had a better chance to survive when treated at an earlier stage (not significant). Most participants of the symposia or Train-The-Trainer meetings already knew that NPC was more common in men than in women (72%); after the trainings, this proportion was 88% (p<0001); at the 1½ years assessment, this was 73% (not significant). Similar results were found for the questions from which age NPC may develop and which age group has the highest incidence (see table 2 and online supplementary table S1).

Supplementary table 1

Overview of the different training programmes

Table 2

General questions regarding nasopharyngeal carcinoma (NPC)

NPC symptoms

The GP invited to the symposium was often the most senior GP working in the healthcare facility. However, no significant differences in knowledge were found between GPs prior to a symposium and prior to a Train-The-Trainer programme (data not shown); these were combined in further analyses. In table 3, model estimates for knowledge increase of symptoms of NPC are presented; the reference is a GP prior to any kind of training with <3 years of work experience. The estimates can be interpreted as absolute percentages. For example, nurses with 3–6 years’ work experience scored 30 (47−18+0.8) points of the 100 prior to training. After attending the Train-The-Trainer programme, the score increased to 57 (47−18+0.8+27) and was 1½ years later still significantly increased at 42 (47−18+0.8+12) points (p<0.001). GPs scored better compared with nurses (−18 points, p=0.008) and midwives (−20 points, p=0.009). The number of years of work experience was negatively associated with knowledge of NPC, but the effect was very small: 3 points on a scale from 0 to 100 for those with 9 years of work experience. Nurses increased their knowledge of symptoms on average with 12 points more than GPs (p<0.001).

Table 3

Symptoms and risk factors: fraction of correct answers given during the different activities adjusted for profession and years of work experience

NPC risk factors

We estimated that before any kind of training the fraction of correct answers regarding the risk factors of NPC for a GP with <3 years’ work experience was 36% (table 3). Again, baseline results of GPs were combined, though GPs completing the questionnaire before the Train-The-Trainer programme scored 1% better than their colleagues before the symposium. After the symposium and Train-The-Trainer programme, this fraction increased to 65% and 62%, respectively. Nurses and midwives had lower scores at baseline compared with GPs, but significantly improved their knowledge from 23 points to 50 points (p<0.001) direct after the Train-The-Trainer programme; the score remained at 40 points after 1½ years (p<0.001 vs baseline). Nurses increased their knowledge five points more than did GPs (p=0.01). An increase in the number of years of work experience was associated with less knowledge of NPC risk factors.

Long-term effects

Forty-one per cent (n=73) of all participants who filled in the questionnaire at 1½ years after the training indicated having counselled NPC suspects. Almost all (72 of 73) were referred, 25 (34%) specifically to an ear, nose and throat (ENT) specialist. As discussed above, knowledge of symptoms and risk factors had overall remained higher compared with before the NPC awareness programme. However, this higher knowledge did not differ between those participants who completed the questionnaire for the long-term effect and claimed never to have followed any kind of training before (76%) and those who did participate in the NPC awareness programme (24%).

Need for additional education

GPs attending the first symposium held in Yogyakarta were also asked what kind of education methods would be sufficient, whether the accreditation is important, and how much time they would like to spend on additional education. In total, 97% of the participants (n=41) agreed that a lecture of an ENT specialist would be an appropriate education; and 47% indicated that personal education by an ENT specialist would be appropriate. Accreditation was important for 90% of the participants and 36% wished to spend >1 day on additional education (see online supplementary table S2).

Supplementary table 2

Interest in additional education


This is the first multicentre study conducted in Indonesia demonstrating that the effect of additional training for NPC symptoms and diagnosis can be successful. Moreover, knowledge of the healthcare workers working in the PHCC was still increased after 1½ years. The long-term effect was only tested in Yogyakarta since the area is more transparent and easy to visit. However, there were no differences in short-term knowledge increase between participants in the different cities. Therefore, we assume that the results obtained in Yogyakarta are representative of those for the other cities where the NPC awareness was conducted.

Considering the training for nurses and midwives, it was not surprising to find lower scores compared with GPs. Nevertheless, in the long term, their knowledge remained improved, suggesting that the programme is effective for different health professions working in the PHCC. It should be noted that in our analyses all observations were treated as independent measurements, and not individually paired scores, because all questionnaires were completed anonymously. This was important to improve the likeliness of honest answers in the questionnaires. Since only GPs were invited to the symposium, we could only test the association between profession and improvement in knowledge for the Train-The-Trainer programme. Indeed, nurses and midwives had less knowledge than GPs, but after the Train-The-Trainer programme it was increased more than the knowledge of the GPs, suggesting that different professions have different learning curves. Taking the role of the nurses in the PHCC into account, it only emphasises the importance of their participation in the programme.

In Malaysia, where the incidence of NPC is also high, Balachandran et al likewise discovered that the knowledge of first-line medical healthcare workers in the PHCC on NPC is limited. They also found that when medical doctors in Malaysia were asked if they thought they had enough knowledge to diagnose NPC, 88.7% of the participants answered no.25 Prasad and Pua noticed a ‘doctors-delay’ of 127 days before diagnosis, acknowledging that there is a need to create more awareness among first-line doctors. However, it is not just the GP in the PHCC who plays an important role in the late presentation at diagnosis. The behaviour of the patient also causes a delay.26 This will need to be investigated in more detail to understand the reasons why patients wait before they seek medical help.

A previous study of Devi et al28 showed that a 2-day training programme was effective for downstaging breast cancer and earlier referral of patients with cervical cancer. Even though this study did not show downstaging for patients with NPC, the number of patients with NPC increased, suggesting a better referral programme. In 2009, an NPC data management system was introduced in Yogyakarta. All new patients with NPC at the Dr. Sardjito Hospital in Yogyakarta are being registered. Future studies can investigate the effect of awareness training on stage at NPC diagnosis.

Seventy-six per cent of those who completed the long-term questionnaire indicated that they had not attended an extra NPC awareness training. This percentage was expected to be smaller, based on the number of years the participant has been working in their PHCC and the interval between the activities. However, we suspect, based on a personal communication with Indonesian colleagues, that one reason for this is that participants do not want to admit that they attended any kind of additional training to prevent losing face when not all questions were answered correctly. In addition, there can be a spillover effect, which proves the importance of the awareness programme. This is not unlikely since all participants of the symposium received training materials, such as folders and flyers, to share with their colleagues. When we visited the PHCCs at the 1½ years assessment, some of them still had an NPC poster on the wall.

Besides strengths, our study also had some limitations. For each training, all healthcare workers working in the PHCC were invited and the training was obligatory. However, since it was not feasible to register non-attendees, there may have been a bias towards more motivated health staff attending the trainings. Importantly, every attendant filled in the questionnaires because this was part of the training and it was obligatory to receive the certificate. Therefore, we expect response bias to be very limited. Another limitation is that we were only able to evaluate the long-term effect of the programme in Yogyakarta. On the other hand, we have no indications to believe that the long-term effect would be different in the other provinces; at least, the short-term effects of the symposia were similar among centres (data not shown). Unfortunately, we were unable to evaluate whether the training has an effect on the timely referral of patients leading to a diagnosis of NPC at an earlier stage. Indonesia does not have a nationwide cancer registry. The referral system is complex and not all suspects of NPC will have been referred directly to one of the hospitals involved. Moreover, in January 2014, the Indonesian government introduced a universal healthcare insurance (Badan Penyelenggara Jaminan Sosial). The referral system is now more regulated: patients first visit primary care and may be referred by the physician to a specialist in a secondary care unit, who can refer the patient to a tertiary care hospital. Therefore, we choose to evaluate the effect of the training programme using an effectiveness measure early in the chain of effects, that is, increased knowledge.


Cancer is increasingly becoming the world's leading cause of death, with an estimated 12.7 million new cancer cases and 76 million deaths in 2008.1 The number of people dying because of cancer is significantly higher in developing countries than in developed countries, owing to the lack of early detection, prevention and limited access to and capacity of sufficient healthcare.29 ,30 Early referral and optimal usage of the limited capacity of available equipment is therefore of major importance.

Access to healthcare facilities may improve for many people by the introduction of the new healthcare insurance and is likely to increase the workload of the GPs in the PHCC. Therefore, it will assume greater importance if GPs are able to recognise the early symptoms and refer the patients to the correct specialist.

Our research shows that the NPC awareness programme is an effective tool to increase the knowledge of the primary healthcare workers. In this way, we wish to achieve diagnosis of NPC suspects at an earlier stage through referral at the onset of the disease. Once the medical specialists are trained, public awareness should be the next step. Patients' attitudes towards the health care system, resulting in a delay in diagnosis, needs to be investigated in more detail before initiating more community awareness.


The authors would like to thank all the participants for their collaboration. Also they would like to thank all Indonesian medical students, public health students and residents who were involved in the organisation of different symposia. Especially Greta Gulo who was closely involved in the data entry and collecting of the data for the follow-up. Their gratitude goes to all Dutch students involved especially Nathalie van den Brekel and Josephine Tan. They also thank to all the different health departments in the districts for their approval and involvement in the organisation. In addition, Donny Artika has been key to keep the overview and supervise the organisation of the different symposia.



  • MKS and IBT shared last and corresponding authorship.

  • Contributors RF initiated and designed the project, participated in the organisation of the symposia, and was involved in the data collection, analysis and writing of the manuscript. SRI was involved in the organisation of the symposia, Train-The-Trainer programme and long-term follow-up and data collection and analysis in Yogyakarta and writing of the manuscript. CH was involved in the organisation of the symposia, Train-The-Trainer programme and long-term follow-up and data collection in Yogyakarta. SM, AI and ACR were involved in the organisation of the symposia and Train-The-Trainer programme and data collection and analysis in East Java. MA was involved in the organisation of the symposia and data collection in Jakarta and the data analysis. IDM was involved in the organisation of the symposia and data collection in Jakarta. EvW conducted the data analysis and was involved in the writing of the manuscript. MAW was involved in initiation and design of the project. BHa initiated and supervised the project in Yogyakarta. BHe initiated and supervised the project in Jakarta. WAK initiated and supervised the project in Surabaya. SMH is a principal investigator of the project, initiated and supervised the project. MKS, a principal investigator of the project, was involved in the data analysis and interpretation of the results and also in the writing of the manuscript. IBT, a principal investigator of the project, was involved in the data analysis and interpretation of the results and also in the writing of the manuscript; all authors read and approved the final version of the manuscript.

  • Funding This project was funded by the Dutch Cancer Society (KWF2012-5423) and Achmea Foundation (2009·34).

  • Disclaimer The funding source had no role in the design, implementation, interpretation and publication of the study.

  • Competing interests None declared.

  • Ethics approval The Indonesian Doctors Association accredited all activities.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement No additional data are available.