Article Text
PDF
Abstract
Objective To translate and validate the consumer information rating form (CIRF) for use in Thai populations.
Design The development of the CIRF was carried out in two phases: translation process and cognitive interview, and psychometric testing.
Setting A university hospital and a tertiary hospital in northeast Thailand.
Participants 150 outpatients from medicine department: 30 for phase 1 and 120 patients for phase 2 study.
Methods The CIRF was translated with cultural adaptation into Thai using cognitive interview technique in a sample of outpatients. A larger sample of outpatients then completed the CIRF in relation to either a package insert (PI) or a patient information leaflet (PIL) for one of three medicines: atorvastatin, celecoxib and metformin. Construct validity was assessed using principal component analysis (PCA) and internal consistency using Cronbach’s α coefficient. Known group validity was assessed by comparing mean consumers’ ratings for PIs and PILs.
Results Thirty participants engaged in the cognitive interview and 120 participants completed the CIRF. The PCA found the 17 items of the CIRF were extracted into three factors: comprehensibility, utility and design quality scales, mirroring the original. Cronbach’s α for the overall scale (0.904) indicated good internal consistency. Known-group validity demonstrated significant differences in consumers’ rating between PIs and PILs for almost all items (p<0.001).
Conclusion Thai version of CIRF had acceptable validity and reliability for Thai consumers’ ratings of written medicine information. The CIRF could be of practical use in the process of developing medicine information to ensure consumers’ comprehension and their usefulness.
- consumer information
- written medicine information
- psychometric analysis
- patient information leaflet
- package insert
Data availability statement
Data are available on reasonable request. The data that support the findings of this study are available from the corresponding author upon reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
- consumer information
- written medicine information
- psychometric analysis
- patient information leaflet
- package insert
Strengths and limitations of this study
This is the first study of translating the consumer information rating form (CIRF) into Thai version using a systematic procedure of translation and cultural adaptation.
The CIRF was validated using a cognitive debriefing technique by different groups of samples to improve understanding about this translated version.
Principal component analysis and known-group validity were used to describe the psychometric properties.
The sample size of this study was enough to perform factor analysis, but larger studies are needed to determine any patient characteristics that might influence opinions of written medicine information.
We only focused on three medications for chronic conditions and did not include any written information for medicines used as short-term therapy.
Introduction
Written medicine information (WMI), a source of essential information about medicines: indications, precautions, directions for use, side effects and storage, plays a vital role in educating patients about medicines.1 2 WMI in the form of patient information leaflets (PILs) has been developed to help ensure that patients have sufficient information to maximise benefits and minimise adverse effects from medicines. PILs are generally designed so that patients are able to read and understand them, making use of simple language and patient-friendly formats. In European countries, before a medicinal product is launched to consumers, a PIL must be produced which meets patient needs and which must then be supplied with every medicine. Testing is required to ensure comprehension of the information in PILs.3 Readability tests are a common method to identify the reading level of written materials by measuring number of words and length of sentences, as a proxy for understanding.4 5 Several formulae have been developed to assess readability of WMI such as the Flesch Reading Ease Score, the Flesch-Kincaid Grade and the Simple Measure of Gobbledygook score.6–8 However, these formulae are applicable only to English text and cannot assess true comprehensibility, which is influenced by various factors including prior knowledge and background of the reader, as well as the appearance of WMI such as illustrations and attractiveness.9 10
It is important that there is patient involvement in assessing WMI. The European Commission, therefore, suggested user-testing to assess comprehension of information in target patient groups. This method has been widely used to test patients’ comprehension in several studies following the Council Directive 2001/83/EC.3 11–13 However, it is not a legislative requirement to use this method to assess patient information in other countries such as Australia.14 The consumer information rating form (CIRF) was developed to measure consumers’ perceptions on three dimensions of WMI: comprehensibility, utility and design quality.15 16 The CIRF has been validated in various groups of consumers, mostly patients on chronic drug treatment16–18 and shown to have high validity and reliability.15 16 Importantly, this instrument has been found effective in other languages such as Flemish patients.19
In Thailand, WMI is a common source of medicine information that is easily accessible in the form of package inserts (PIs) which are enclosed with medicinal products. However, PILs are only available for some medicines. PIs are designed for use by health professionals, so include large amounts of technical information, using medical terminology, in contrast to PILs, which are designed for patient use and contain less detailed information in lay language. User-testing is the only one recommended by the Thai Food and Drug Administration (FDA) as an effective method to identify problems with and help improve Thai PILs.20 A previous study has shown that user-testing was feasible in a Thai population and could help improve the quality of Thai PILs,21 although it is not widely applied. The CIRF could become a second method for assessing Thai WMI either alone or in combination with user-testing. This study aimed to translate the CIRF into Thai and assess its validity and reliability.
Methods
Study design and setting
This study was conducted in Khon Kaen, northeast Thailand, during April to October 2020. The study consisted of two phases: Translation and cognitive interview (phase 1) and psychometric testing (phase 2) (figure 1).
Diagram of study process. CIRF, consumer information rating form.
Study instrument
The original CIRF (17 items) consists of three main sections exploring consumer comprehensibility (5 items), utility (6 items) and design quality of WMI (6 items). The five comprehensibility items explore the ease of the WMI to read, understand, locate information, remember and keep for future reference, measured by 1–5 Likert scale. The utility was the composite scale of perceived quantity and usefulness of the WMI, measured by 1–3 scale. The design quality items explore perceptions about design, layout and tone of the information, measured by 1–5 Likert scale.15 The original CIRF was subsequently adapted by adding questions about the likelihood of using the WMI in the future and one item regarding bias into the design quality section.16
Participants
In phase 1 (cognitive interview), participants were recruited in the outpatient medicine department of Srinagarind Hospital and Queen Sirikit Heart Centre, Khon Kaen, Thailand during July to August 2020. Patients who were at least 18 years old, currently taking one of the three prescription medicines atorvastatin, celecoxib or metformin were invited to join the study. These three medicines were frequently prescribed in Thailand and used in long-term treatment. Written information about these medicines was available in the form of PILs supported by the Thai FDA and the PIs supplied by the manufacturers. Patients who were not willing to join the interview and those who had any problems with reading and communication were excluded.
In phase 2 (validation), patients who visited either medical outpatients or the pharmacy department, fulfilled the same inclusion criteria and did not participate in phase 1 were invited to take part. The participants were selected to ensure diversity in both the type of medicines used, which were atorvastatin (n=40), celecoxib (n=40) and metformin (n=40), and the type of received WMI, either PILs (n=60) or PIs (n=60).
Study procedures
Phase 1: translation process and cognitive interview
Permission to use the original CIRF was obtained from Krass et al.15 Translation of the CIRF from English to Thai followed principles of good practice for the translation and cultural adaptation process for patient-reported outcomes measures.22
Step 1: forward translation
Two Thai translators, who were pharmacists and had expertise in health research (KW and JP) independently translated the CIRF into Thai using clear wording and literal meanings.
Step 2: forward translation review
A translation panel consisting of the two independent translators (KW and JP) and one principal investigator (NJ) discussed both versions of the translated CIRF and merged them into a first translated version (CIRF-Thai V.1). The goal of translation was to choose the words and phrases that closely reflect the original version as much as possible.
Step 3: back translation
The translated CIRF was translated from Thai into English by another translator who had never seen the original CIRF (JP).
Step 4: back translation review
Both original CIRF and back-translated versions were compared by a native English speaker (JK) to assess the conceptual equivalence of the translation. If there was a lack of clarity about items in the two versions, the research team re-ensured this conceptual equivalence with the original developer. Findings from discussion at this stage were used to edit the CIRF-Thai V.1–V.2.
Step 5: cognitive debriefing
The WMI (either a PI or PIL) and the CIRF (Thai version) were given to potential participants by the researcher. To ensure similar numbers of the PIs and PILs received and diversity in types of medicines, participants were selected by quota sampling so that 30 participants were selected based on types of medicines (atorvastatin=10, celecoxib=10 and metformin=10) and types of WMI (PILs=15, PIs=15). The three medicines used in the study have a PI inserted in the medicine box and also a PIL which is available on the Thai FDA webpage.20
The participants were asked to read either the PI or PIL, and the CIRF (Thai version) starting from the instructions, questions and response options thoroughly. Cognitive interview using a think-aloud technique was conducted by a researcher (KW) and notes were taken. Participants were asked to express their understanding of each question and response options and the meaning of the words of the CIRF-Thai version. They were asked to evaluate the length and difficulty of the CIRF and were also asked to provide any suggestions to make changes to improve the word clarity, instructions, the difficulty of completion, length and usability of the CIRF-Thai version in evaluating WMI. Findings from the cognitive interviews were used to improve the CIRF-Thai V.2 leading to the final V.3.
Phase 2: psychometric testing and data analysis
The final version of the CIRF-Thai version was distributed to outpatients by only one researcher to avoid bias. The first participant was randomly assigned to read either a PI or PIL, selected at random, then then the second participant received the other leaflet, using alternating allocation.
However, if any participants assigned to read the PI were unable to read it clearly because of too small text, they were asked to read the PIL instead.
Construct validity
The construct validity of the CIRF was explored using principal components analysis (PCA). Oblique rotation (varimax) was considered for component interpretation. The Kaiser-Meyer Olkin (KMO) and Bartlett’s test of Sphericity values were calculated to examine the adequacy of sample size and suitability of data,23 taking a KMO value >0.8 and p<0.05 for Bartlett’s test as demonstrating that the data were sufficient to undergo factor analysis.24 To examine the components of the CIRF, Kaiser’s eigenvalue greater than 1.0 and the scree plot were determined. Any items with poor factor loadings (<0.35), low communalities (<0.3) and cross-loading (>0.4) on two or more factors were considered for removal.25
Internal consistency
The internal consistency was examined using Cronbach’s α to assess the relationship between items, using a value of >0.7 as acceptable. The CIRF was considered as reliable with the value greater than 0.7.26
Known-group validity
Known-group validity was assessed by comparing patients’ scores of the CIRF-Thai version for PILs and PIs. It was hypothesised that opinions about PILs would be more positive, compared with PIs in at least one of the CIRF components: comprehensibility, utility and design quality. It was not expected that age or gender would affect opinions, although educational level could potentially affect opinions on comprehensibility.
Views of using CIRF
The views of participants regarding completing the CIRF were obtained after they finished completing the CIRF. Views were determined using five positive and five negative statements using 5-point Likert-type Scale with responses ranging from strongly disagree (1), disagree (2), not sure (3), agree (4), to strongly agree (5).
Statistical analysis
The sample size for validation was determined on the basis of the number of responses per item. As there is a lack of clear number of sample size for patient-reported outcomes measures,27 seven subjects per item were justified in this study. Data were analysed using SPSS for Windows V.23.0. Demographic data of participants was presented by simple frequencies. Participants’ rating scale regarding PIs and PILs were presented using mean and SDs. Pearson χ2 and Fisher’s exact test were used to compare demographic characteristics of participants reading PIs and PILs. The difference between their ratings towards PIs and PILs were determined using independent sample t-tests. Views of participants towards completing the CIRF were calculated by first reverse scoring responses to negative questions and summing scores, then these were presented in mean±SD and were classified into three equal categories: negative (10–22 points), neutral (23–36 points) and positive view (37–50 points).
Patient and public involvement
Patients or public were not directly involvement in the development of the research design, concept or data interpretation of the study.
Results
Phase 1: translation and cognitive interview
Translation
Overall, the two forward translations had the same meaning, with only minor differences. Interpretation of the word ‘benefits’ could be general benefits of taking the medicine or it could be indications in a pharmaceutical context. The word ‘tone’ could be interpreted as the tone of language used to make readers feel worried and alarming or tone in the extent to which colour was used on the leaflets. The panel agreed that the interpretation should be the benefits of taking the medicine, and tone in relation to language used in WMI. Other inconsistencies between translations were resolved by choosing the one closest to the original version.
The back-translation into English was almost the same as the original. Nine discrepancies were identified by the expert panel, which concerned slight differences in the word used, with similar meanings. Minor changes were made in this stage. The word ‘essential’ was removed from ‘essential precautions during the use of this medication.’ The term about the organisation of the information sheet was raised as potentially ambiguous, having been translated into layout in Thai. To resolve the inconsistency, the translation panel asked the original developer, and this term was corrected to ordering of the information, not layout of the information sheet.
Cognitive interview
Forty-five patients were approached to be interviewed, of whom 15 refused to join the study. Thirty participants were interviewed. Demographic characteristics are shown in table 1. Around half of participants (n=16, 53%) were female. The majority of participants (n=21, 70%) had a bachelor’s degree or higher.
Patient characteristics participating in cognitive interview (n=30)
Participants considered the CIRF in respect to the understanding of instructions, questions and response options. Nineteen participants agreed that the CIRF was appropriate in length, and 24 agreed that the CIRF was organised to read easily. The difficulty of completing the CIRF was moderate. Twenty participants considered that the questions of the CIRF were suitable for assessing WMI.
About two-thirds of participants did not understand the meanings of all the words presented in the CIRF. Several changes were required to improve the clarity of the instrument and enable self-completion. In the comprehensibility subscales which used the instruction to select ‘how easy or hard’ aspects of the information sheet were, the option ‘remember’ was not judged to be clear. Participants were unsure what they should remember about and felt the response options did not match with the question ‘how easy or hard’ leading to difficulty in completing this item. The word ‘remember’ was, therefore, changed to ‘ability to remember the information’. In the utility subscale which covered how much information and how useful information was, the phrase ‘who should not use the medication’ was considered vague in a Thai context. Many participants questioned what did ‘who’ in this phrase mean and they suggested that the phrase should be replaced by an alternative such as ‘person that should not use this drug’. Also, many participants did not understand ‘what to do about them’ in the phrase ‘possible side effects and what to do about them.’ They frequently mentioned it was not appropriate because it was not normally used in Thai written language. Therefore, this phrase was edited into ‘possible side effects and practice to do when the side effects occur’ which was more polite and formal.
In the design quality subscale, it was clear that some words had more than one meaning. Some considered that the word ‘bias’ could be interpreted as their own prejudice towards the information written on the sheet, positive or negative, while others considered it concerned how information was written in a positive or negative way, too much emphasis on benefits or negative information. The research team agreed that the word “bias” should be replaced by ‘equality of the information’.
In addition to word changes, some changes in the design of the CIRF-Thai version were made, including adding border lines to all tables. Several participants, particularly elderly people, reported a difficulty answering the CIRF due to no leading lines. Descriptions of rating numbers 1–5 of the design quality questions were added as follows: score 1 means this WMI is bad in terms of organisation, print size, attractiveness, tone, helpfulness, bias and spacing between lines, and score 5 means this WMI is excellent in all these aspects (please see at online supplemental material 1).
Supplemental material
Phase 2: psychometric testing
Demographic data
A total of 171 patients were invited in this stage. Of these, 19 patients were excluded because they could not read the PILs or PIs clearly, and 21 patients were not willing to join the study. Eleven patients did not complete all items in the CIRF. Thus, data from 120 patients were included for analysis, 60 each assessing a PI and a PIL. The characteristics of participants are presented in table 2. Six participants were switched from PI group to PIL group due to eyesight problem. More than half of the participants were female (59.2%) and had bachelor’s degree or above (58.3%). Most participants had health insurance in Civil Servant Medical Benefits Scheme (77.5%). Overall, there were no statistically significant differences between patients reading PIs and PILs except for age group, with those who mostly read PIs were age 45–60 while those who most read PILs were older than 60 (table 2).
Patient characteristics in psychometric testing categorised by types of WMI (n=120)
Construct validity
The KMO value was 0.879 and Bartlett’s Test of Sphericity was significant (p<0.001) indicating that the data set was sufficient for factor analysis. PCA suggested three-factor components regarding consumer rating about WMI, aligning with the original CIRF. They were the consumer utility rating scale, consumer design quality rating scale and consumer comprehensibility rating scale that explained 41.024%, 13.976% and 11.431% of variance, respectively. There was neither any items with poor factor loadings, nor cross-loading on two factors. No items showed low communalities. The factor loadings are presented in online supplemental material 2.
Supplemental material
Internal consistency
Reliability analyses revealed good internal consistency in overall scales and subscales. Cronbach’s α coefficients were 0.884 for the consumer utility rating scale, 0.880 for the consumer design quality rating scale, 0.896 for the consumer comprehensibility rating scale and 0.904 for the overall scale.
Known-group validity
The comparison between patients’ rating regarding PIs and PILs revealed significant differences in all items within the comprehensibility, and design quality domains, as well as future use, and almost all items in the utility domain except contraindication information. All results are shown in table 3.
Consumer rating of WMI using the consumer information rating form
Views of using CIRF
A large majority of respondents agreed or absolutely agreed that they were satisfied with using the CIRF form (89.1%), that it was easy to read and complete (90%), and well organised (82.5%). However, about half of the participants considered they may need other people to help them complete the CIRF form. Most respondents agreed or absolutely agreed that the CIRF form was useful for consumers to assess information about medicines (90.8%). In contrast, over half of respondents disagreed that the contents of the CIRF were not suitable (68.4%), and the numbers used in the form were difficult to understand (51.6%). Responses to the ten statements regarding using the CIRF are presented in table 4.
Views of patients using the consumer information rating form
The overall mean score was 36.10±4.85 (lowest-highest score=23–50). About half of the participants (54.2%) had neutral viewpoint (mean=32.58 ± 2.90) and the remaining participants had positive viewpoint of completing the CIRF (mean=40.25 ± 3.07).
Discussion
This study aimed to translate the CIRF into Thai and validate it as an instrument for assessing WMI from the patients’ perspectives. The CIRF validation process involved assessing construct validity using PCA, known group validity and internal consistency. The internal consistency of the Thai-CIRF was acceptable with all the components exceeding the accepted α value of 0.7. The 17 items of the Thai-CIRF were factorised into three components: comprehensibility, utility and design quality, which were consistent with the factors identified in the United States, Australian and Belgian studies.15 16 19 The total variance explained by these factors was 66.431% in our study, which was slightly lower than the US study was (79.314%),15 but more than the studies from Australia and Belgium (59.30% and 52.97%, respectively).16 19 It could be explained by smaller sample sizes having been used in the US and Belgium studies.15 19
The CIRF has been used in the evaluation of consumer medicine information in Australia and medicine pamphlets designed by hospital pharmacists in Belgium.16 19 The CIRF was also used to compare Australian consumers’ ratings between two forms of PILs; standard community pharmacy PILs and model PILs which were developed by the researchers. This latter study showed that the CIRF could be used to differentiate the perceived comprehensibility, utility and design quality of the two forms of PILs.15 This study compared patients’ ratings regarding two types of WMI; PIs and PILs, finding considerable differences between ratings, thus supporting known-group validity of the Thai-CIRF.
A patient-centred framework for effective communication, which could be applied to PILs, was developed in the UK in 2011. This stated that three components: readability, comprehensibility and communicative effectiveness were related, all were necessary, and that the framework could be used as guidance for PIL development.28 While many different readability tools exist and are widely used to assess WMI, they are not derived from a patient perspective. Several tools have been developed to assess users’ perspectives regarding WMI, but not all cover all aspects of this framework. The Satisfaction with Information about Medicines Scale assesses the extent to which information meets patients’ need.29 User testing, widely used in European countries, is effective in assessing patients’ comprehension and ability to find essential information from the PILs.11 13 30 The design quality of WMI could be assessed by the Suitability Assessment of Materials31 and the Medication Information Design Assessment scale.15 Subjective usefulness towards WMI could also be assessed by the Usefulness Scale for Patient Information Material.32 The CIRF has been demonstrated as an effective tool to evaluate patients’ perspectives in three dimensions: comprehensibility, utility and design quality in various populations.15 16 19 33 However, none of these tools can measure successful communication. The communicative effectiveness of WMI (covering expectations, concerns and behaviours) still needs to be explored and relationships among readability, comprehensibility and patient-related outcomes evaluated.28
A cross-sectional study in Thailand exploring consumers’ opinions on PIs reported reasons for reading and not reading a medicine leaflet using factor analysis. This study found four factors: difficult content, dual ideas, small print and satisfaction, explaining why Thai consumers did not read the PIs,34 which is in accordance with the comprehensibility and design quality domains of the CIRF. A user-testing study in 2019 which assessed PILs developed following Thai FDA guidelines found that some participants were not able to locate important information from the PILs due to the words used in topic headings and that text size needed to be larger than guidelines require.21
This was the first study on translating the CIRF into Thai following a systematic procedure and exploring the initial psychometric properties in Thai patients. This study has several limitations. First, it was only conducted in one region of Thailand; hence, larger studies are needed to determine any patient characteristics that might influence opinions of WMI. Second, this study focused on the written information available for only three long-term medications and did not include any written information for medicines used as short-term therapy. For future research, evaluation of knowledge between reading PIs and PILs needs to be conducted to compare patients’ comprehensibility and their usefulness between the two types of information.
Policy implications
Consumers have the ability to assess the quality of WMI in terms of their comprehension, and the usefulness, and presentation of the information.5 16 30 Consumer involvement in evaluating medicine information is now recommended across the European Union.30 Consumer evaluation of WMI has benefits to improve WMI to be more comprehensible and suitable for reading. However, there has been no gold standard to evaluate the quality of WMI. The CIRF could be a potentially useful instrument that could be applied in a variety of settings, to develop effective WMI. It can be used in combination with other methods such as user testing to cover consumers’ perspectives more comprehensively.
Conclusions
Consumer evaluation is an important process to develop effective medicine information. The Thai version of CIRF has acceptable validity and reliability. It is also consistent with construct validity when compared to the original English version. Using the CIRF in assessing Thai WMI could enable medicine information creators to gain further insight about what consumers perceive about the comprehension of information, utility and design quality of currently available WMI.
Data availability statement
Data are available on reasonable request. The data that support the findings of this study are available from the corresponding author upon reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
This study was approved by the Khon Kaen University Ethics Committee for Human Research (number HE611500) and conducted in accordance with the ethical principles of the Declaration of Helsinki. All potential participants were informed of the study objectives and verbal consent was obtained; the process was approved by the Ethics Committee.
Acknowledgments
We thank the Royal Golden Jubilee Ph.D. Programme by Thailand Research Fund for supporting this research (Grant No. PHD/0117/2559). We appreciate Professor Ines Krass, for permission to use the CIRF and for providing valuable suggestions to our study, and Dr. Pacharaporn Phueanpinit for cooperating in the translation process. We also thank outpatients from Srinagarind Hospital and Queen Sirikit Heart Centre of the northeast who participated in the interview and to all staff who provided help in data collection.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors KW, JK, JP and NJ: concept and research design. KW and SP: data collection. KW: formal analysis and interpreting data, drafting of the manuscript. JK and NJ: critical revision of the paper for important intellectual content. NJ, JK and JP: supervision.
Funding This research was financially supported by the Royal Golden Jubilee Ph.D. Programme Scholarship (Grant No. PHD/0117/2559) by Thailand Research Fund.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.