Article Text
PDF
Abstract
Objective It is crucial to use clinically validated instruments to detect and treat depression in people with epilepsy. Therefore, this study aimed to describe the psychometric properties of the Afaan Oromo version of the Hamilton Depression Rating Scale 17-item (HAMD-17) among these individuals in Ethiopia.
Design A cross-sectional validation study.
Setting A tertiary hospital, Southwest Ethiopia.
Participants A total of 133 people with epilepsy were included in this study using a consecutive sampling technique from 1 September 2020 to 30 September 2020.
Outcome The psychometric property of the HAMD-17. Criterion validity was assessed using the Mini-International Neuropsychiatric Interview (MINI). The internal consistency was determined using Cronbach’s alpha. The receiver operating characteristic (ROC) analysis was used in determining the cut-off score, sensitivity, specificity and positive predictive value (PPV) and negative predictive value (NPV). The tools were translated into the local language (Afaan Oromo) and back into English and pretested before the data collection.
Results The mean age of the participants was 31.7 years, SD±10.7. Eighty-six (64.7%), 55 (48.1%), 68 (51.1%) and 62 (46.6%) of them were male, unmarried, urban residents and completed primary school, respectively. The internal consistency of HAMD-17 was α=0.74. The Pearson’s correlation coefficient for criterion validity was r=0.88. The ROC analysis showed 0.92 and 0.91 sensitivity and specificity of the HAMD-17, respectively. It also revealed α=0.96 NPV and α=0.87 PPV at a cut score of ≥9. The area under the ROC curve was 0.96.
Conclusions The Afaan Oromo version of the HAMD-17 was considered valid and reliable for assessing depression in people with epilepsy, and the tool is not difficult to understand. The diagnostic performance using the gold standard MINI showed a good discriminatory capacity of the instrument. It can be used by any health professional to screen depression in people with epilepsy.
- psychiatry
- depression & mood disorders
- epilepsy
Data availability statement
Data are available on reasonable request.
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
It was conducted only among people with epilepsy; hence, applying it to other individuals with chronic medical illnesses may require additional consideration.
As the study was cross-sectional, validity and reliability tests, such as predictive validity, test-retest reliability and factors that affect the score, were not possible.
Given the research question examined the psychometric properties of Hamilton Depression Rating Scale 17-item, the sample size of this study is a bit smaller but met the objectives.
Test-retest reliability was not conducted to correlate the scores to evaluate the test for stability over time due to the limitation of the resource.
Introduction
Depressive disorder is a common mental disorder that negatively affects an individual’s feelings, thinking and behaviour.1 It is a common comorbid disorder in several neurological disorders,2 particularly in epilepsy ranging from 20% to 55%.3
According to neuroimaging and neurobiological studies, the biological description of this linkage is based on neurochemical and neuroanatomical principles.4 Moreover, the relationship between the two disorders may be due to comorbid psychosocial stressors.5
A population-based study in Ethiopia showed that depression is strongly associated with mortality and disability, with a high chance of experiencing >15 disability days per month. The prevalence of depression in the study in Ethiopia was 26%.6
Another study conducted in Northwest Ethiopia revealed that the point prevalence of depression in people with epilepsy was 45.2%, of which 29.6% were mild, 14.8% moderate and 0.8% severe.7
Despite depression being highly prevalent in people with epilepsy, it continued to be underdiagnosed and untreated in these people.8It has been shown that up to 52% of people with epilepsy with comorbid depression were inadequately treated and often not diagnosed.9 As a result, depression makes epilepsy more difficult by complicating the morbidity, mortality and quality of life.10
Screening for depression for people with epilepsy resulted in a 10 times higher rate of diagnosis, which highly enhances early detection and treatment of the disorder in epilepsy.11 However, currently, there are a limited number of validated depression assessment instruments for people with epilepsy.10 12–14
The Hamilton Depression Rating Scale 17-item (HAMD-17) is a popular instrument commonly used by clinicians and researchers. It is the most widely used instrument to measure depression and was developed by Max Hamilton in the late 1950s and published in 1960 to assess the effectiveness of first-generation antidepressants.15 16 The original HAMD-17 included 21 items, but 4 of the last items should not be taken into account as they do not indicate the severity of depression rather than providing additional clinical information.16
The HAMD-17 was preferred to be validated to screen depression in people with epilepsy due to its applicability in less-educated people and the use of clinical data collected from a more objective assessment of symptoms which involves non-verbal feeling states through facial expression, posture, voice and overall appearance by well-trained clinicians. In contrast to the self-ratings instruments, the clinician-administered nature of HAMD-17 could minimise the dependence on patients’ subjective judgement, which may lead to over-reporting or under-reporting of symptoms. The tool also assesses biological symptoms of depression extensively.10 17
This tool was used to assess patients’ levels of depression before, during and after treatment. Based on the HAMD-17, scores from 1 to 7, 8–16, 17–23 and >24 were considered normal, mild, moderate and severe depression, respectively.18 The scale has been validated for people with epilepsy in various languages and cultures.10 14 However, it has not been validated in most sub-Saharan African countries, including Ethiopia.
Because detection is the stepping stone for appropriate treatment that increases the quality of service,19 a valid and reliable tool is required for people with epilepsy to overcome the impact of depression. This is also crucial to validate the Afaan Oromo version of the HAMD-17 to improve the quality of life in people with epilepsy who manifest symptoms of depression. Thus, the current study aimed to validate the psychometric properties of the HAMD-17 Afaan Oromo version to detect depression in people with epilepsy in Ethiopia.
Methods and materials
Study setting and period
The study was conducted at the Jimma University Medical Center (JUMC), Jimma town, Oromia regional state, Southwest Ethiopia. Jimma town is 352 km from Addis Ababa, the capital city of Ethiopia. Jimma University has made relentless efforts in extensive renovation and expansion work to make JUMC conducive for clinical service, teaching and research activities.20 Currently, 1523 adults with epilepsy are receiving services in the neurology clinic of the centre. The study was conducted from 1 September 2020 to 30 September 2020.
Study design
A cross-sectional validation study was conducted, and the Strengthening the Reporting of Observational Studies in Epidemiology guideline was used.21
Sample size determination
The sample size was determined using a single validation test formula, considering 80.6% anticipated sensitivity and 82.5% anticipated specificity from the study conducted in Lebanon.22 According to the study conducted in Jimma among people with epilepsy, the prevalence of depression was 49.3%.23 Finally, 133 participants were selected to be involved in the study.
Sampling technique
The study participants who fulfilled the inclusion criteria were recruited using a consecutive sampling technique out of 1523 adult people with epilepsy who were getting service in the neurology clinic of JUMC until the required sample size was fulfilled. People with a confirmed diagnosis of active epilepsy by neurologists who can speak Afaan Oromo fluently and an age of ≥18 years were included in the study. People with epilepsy, who were suffering from severe seizures, and were unable to communicate during the data collection period, were excluded.
Data collection procedures
Two BSc psychiatry professionals and two clinical nurses collected data through face-to-face interviews. The information on sociodemographic characteristics such as age, sex, religion, ethnicity, education level, marital status, occupation and current residency was obtained by directly interviewing people with epilepsy. The data on the diagnosis of epilepsy, type of epilepsy and duration of illness and antiseizure medication treatment profile were collected through an extensive review of the records from their chart. The interviews were interchangeable after every case to reduce the item-order effect. The first participant was initially interviewed by a clinical nurse and then by a blinded psychiatry professional using the HAMD-17 and Mini-International Neuropsychiatric Interview (MINI), respectively. The next participant was interviewed by a psychiatric professional using the MINI and then continued vice versa. The order continued in the same manner for all participants. After administering the MINI, the psychiatry professionals made an overall clinical judgement about whether the participant had depression.
Data quality control
The tools were translated into the local language (Afaan Oromo) and back into English before the data collection period. The data collectors were trained for 1 day before the actual data collection time on each item of sociodemographic factors, the MINI and HAMD-17 of the questionnaires. The training also focused on the purpose of the study, issues related to the consent and privacy of participants, detailed information about the tools, familiarity with how to fill the tools and keeping the order of administration. The data collectors were oriented with approximately 45 min to finish the interview, particularly about 15 min to complete the MINI and 20 min for HAMD-17. A pretest that encouraged comments on the acceptability, clarity and cultural equivalence of the HAMD-17 was conducted on 10% of the sample at the Agaro hospital. The supervisors provided regular supervision and support for the data collectors. The collected data were checked and reviewed for missing data, completeness and consistency before entry.
The MINI was used as a gold standard tool in this study. It is a short, simple, highly sensitive and specific structured diagnostic interview designed to assess major psychiatric disorders in Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). It was introduced by psychiatrists in collaboration with clinicians in the USA and Europe. The MINI was intended to meet the gap for a brief but accurate structured psychiatric interview in research and clinical settings. Non-specialist interviewers can administer the MINI easily. The MINI helped clinicians attain an accurate diagnosis and well-accepted clinical assessment for depression.24–27
Data processing and analysis
After checking the data for completeness and consistency, they were coded and entered into the computer using Epi Data V.3.1 and then exported to the SPSS V.23 for analysis. Sociodemographic characteristics were analysed using descriptive statistics.
The internal consistency reliability of the HAMD-17 was determined using Cronbach’s alpha, and the inter-rater reliability of the tool was calculated by 20 participants independent of the sample size. Sensitivity, specificity and area under the curve (AUC) for various HAM-D17 cut-off scores were calculated using receiver operating characteristic (ROC) analysis. The criterion validity of the HAMD-17 against the MINI was assessed using the Pearson’s correlation coefficient.
Semantic validity
An independent team of bilingual experts in mental health and Afaan Oromo linguistics from different universities translated the HAMD-17 into Afaan Oromo and back into English, respectively (online supplemental appendix 1). The latter experts were blinded to the initial translation.
Supplemental material
Content validity
To assess the conceptual equivalence or content validity of the HAMD-17 scale, the translated version of the tool was pretested on 20 participants before the actual data collection time. The following criteria were used to identify problematic areas during the pretest.
The first requirement was when the respondent disclosed that the meaning of the item was unclear. The second was when participants responded to the questions but could not explain what they understood from them. The last was when respondents gave examples that indicated misconceptions of what the question was intended to elicit. Then the discrepancy between semantic and conceptual equivalence was solved by involving both forward and backward translators. Accordingly, the final translated items were used for data collection. Three experts used the item-level content validity index (I-CVI) (one psychiatrist and two MSc mental health professionals).
Patient and public involvement
Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Results
Sociodemographic characteristics
One hundred thirty-three participants were involved in the study. They ranged in age from 18 to 63 years, with a mean of 31.7 (SD±10.7) years, and 86 (64.7%) being male. About 112 (84.2%) and 12 (9%) respondents were Oromo and Amhara, respectively. Of the total participants, 55 (48.1%) were not married, 100 (75.2%) were Muslim, 42 (31.6%) were farmers and 68 (51.1%) were living in urban areas. Regarding educational status, 62 (46.6%) and 33 (24.8%) had completed primary and secondary school, respectively (table 1).
Sociodemographic characteristics of people with epilepsy visiting the neurology clinic at the Jimma University Medical Center, 2020 (n=133)
Prevalence of depression in people with epilepsy
In this study, 42 (31.6%) of people with epilepsy were identified as experiencing depression using the HAMD-17 tool, and 37 (27.8%) were diagnosed with depression by the MINI.
Semantic validity
The forward translation to the Afaan Oromo Language of the HAMD-17 was performed by two independent bilinguals, Afaan Oromo and English. One is MSc professional in mental health specialty from Mettu University, and the other is an MSc student in mental health from Jimma University. Backward translation (to English) of the HAMD-17 Afaan Oromo version was performed by two independent bilingual translators from the Jimma University Department of Afaan Oromo Language. The Afaan Oromo version of HAMD-17 revealed no discrepancy from the original version. The translation team (psychiatry professionals and Afaan Oromo linguistics) conducted a further investigation to identify difficult items for translation. Finally, they concluded that the current version of the tool was understandable and culturally acceptable.
Criterion validity
The criterion validity of the HAMD-17 was determined using the ROC analysis. The stated score of the AUC, against the gold standard MINI of DSM-5, was 0.96 (95% CI 0.94 to 0.993) (figure 1). The bivariate correlation between HAMD-17 and MINI also showed a statistically significant correlation (r=0.88). In our study, the optimal cut-off point for HAMD-17 was ≥9 in people with epilepsy. At this cut-off score, a sensitivity of 92% (95% CI 0.78 to 0.96) and a specificity of 91% (95% CI 0.81 to 0.96) were obtained, respectively. The positive predictive value was 87%, whereas the negative predictive value (NPV) was 96%. With this cut-off score, maximum sensitivity (92%) and NPV (96%), the Afaan Oromo version of HAMD-17 showed adequate screening properties. The positive (LR+) and negative (LR−) likelihood ratios at this cut-off point with the Youden Index (0.83) were 10.22 and 0.08, respectively. Higher values of LR+ indicate markedly better information values for a diagnostic test, whereas lower values (close to 0) of LR− have better information values for a negative test.
Receiver operating characteristic (ROC) curve for the Hamilton Depression Rating Scale 17-item in people with epilepsy visiting the neurology clinic at the Jimma University Medical Center, 2020 (n=133). The area under the ROC (AUROC) curve, 0.966 (95% CI 0.94 to 0.993), p<0.000.
For the different cut-off points of the HAMD-17, the sensitivity, specificity and positive predictive value and NPV were calculated. The optimum cut-off point was determined using the maximum value of the Youden Index ((sensitivity+specificity)−1) (table 2). Cut-off points ≥9 showed a higher Youden Index of 0.83. As shown in the table, when the cut-off score decreased to 8, the Youden Index value decreased to 0.77. Again when the cut-off point was increased to 10, the Youden Index value decreased to 0.73.
Sensitivity (SN), specificity (SP), positive predictive value (PPV) and negative predictive value (NPV) at different cut-off scores for the Hamilton Depression Rating Scale 17-item in people with epilepsy visiting the neurology clinic at the Jimma University Medical Center, 2020 (n=133)
Content validity
The content validity of the translated Afaan Oromo version of the HAMD-17 scale was assessed on 20 study participants, and 14 (70%) of them revealed difficulty in understanding items 1 and 7. Item 1 (miira gaddaa) and one of its entities, ‘gargaarsa dhabuu’, were difficult to understand despite the translations being explicitly forwarded and requiring a more thorough explanation. As a result, item 1, ‘depressed mood’, was translated as ‘miira gaddaa’, which means ‘feeling sad’, together with the additional option ‘gammachuu dhabuu’, which means ‘feeling unhappy’. Item 1 entity ‘helplessness’, which was translated as ‘gargaarsa dhabuu’, understood as if they were asked for financial difficulty (loss of financial help). Therefore, the term ‘helplessness’ was translated as ‘miira maxxantumma’, which means ‘feeling of dependency’.
In item 7, ‘work and activities’ (dalagaa fi gochoota), although the translation is forwarded distinctly, there was a conceptualising problem. So, it was changed to ‘loss of interest for work and activities’ and interpreted as ‘dalagaa fi gochootaaf fedhii dhabuu’. Following these adjustments, the participants understood the items without difficulty.
Additionally, to ensure the content validity of the HAMD-17, this study used the CVI, the most commonly used method to calculate content validity statistically. Out of the two types of CVI, the current study used I-CVI, which was computed as the number of experts giving a rating of ‘3 (quite relevant) and 4 (highly relevant)’ for each item and divided by the total number of experts (I-CVI=agreed items/number of experts). In rating for relevancy, a 4-point Likert scale was used with 1, 2, 3 and 4 indicating not relevant, somewhat relevant, quite relevant and highly relevant, respectively. Ratings 3 and 4 indicated valid content, whereas ratings 1 and 2 were considered invalid.
One psychiatrist and two MSc professionals in mental health who were bilingual experts were involved in the content validity. The Afaan Oromo version HAMD-17 was sent to these experts via their email addresses to identify the measurement aim of the questionnaires, its clarity, the concepts that the questionnaire is intended to measure and the interpretability of the items. The experts’ responses were collected, and the calculated I-CVI for each item showed high content validity of individual items (I-CVI=1.00), which indicated excellent28 content validity (table 3).
The relevance ratings on the Afaan Oromo version of the Hamilton Depression Rating Scale 17-item scale by three experts, 2020
Face validity of HAMD-17
The Afaan Oromo version of HAMD-17 demonstrated good face validity when the experts asked for the tool on whether the items were unclear, conceptually indicated different meanings or did not capture what was meant to be measured. Moreover, throughout the study, no non-responding participants with a high item-completion rate revealed strong face validity.
Reliability of HAMD-17
The internal consistency reliability for the HAMD-17 was Cronbach’s alpha=0.747, which increased to 0.748 when the 14th (genital symptoms) item was removed. The interitem correlation matrix results revealed that it ranged from a low score of −0.076 between item 3 (suicidality) and item 14 (genital symptoms) to a high score of 0.563 between items 1 (depressed mood) and 2 (feelings of guilt). From the analysis, item 1 achieved the highest correlation (0.507) with the scale. The corrected item-total correlation score ranged from 0.122 (item 14—genital symptoms) to 0.507 (item 1—depressed mood) (table 4).
The scale mean if item deleted, scale variance if item deleted, corrected item-total correlation, squared multiple correlation and Cronbach’s alpha if item deleted for Hamilton Depression Rating Scale 17-item among people with epilepsy, Jimma University Medical Center, 2020 (n=133)
The inter-rater reliability of HAMD-17 demonstrated an excellent measure of agreement between the two data collectors with a kappa coefficient of 0.88.
Discussion
The prevalence of depression was 31.6% with HAMD-17 and 27.8% with MINI, the gold standard tool, in this validation study of the Afaan Oromo version of HAMD-17 in individuals with epilepsy. The disorder was a bit increased while assessed by HAMD-17. However, the prevalence of depression found in this study was lower than that of a study conducted in Nigeria (42%) in people with epilepsy.29 This discrepancy could be because of the difference in study design and utilisation of different versions of the tool in which the previous studies were conducted by case-control using HRSD-21 items, yet our study was carried out by cross-sectional method using HAMD-17. With all this, the current study used cut-off scores of ≥9 of the HAMD-17, and the Nigerian study used cut-off scores of ≥11 to identify people with depression. On the other hand, the reported prevalence of depression in our study was within a similar range to what was found by other researchers from Ethiopia and Egypt.7 30 31
This study showed acceptable internal consistency with Cronbach’s alpha 0.747 of psychometric properties for the Afaan Oromo version of HAMD-17 in people with epilepsy. This finding was within a similar interval with the previous studies conducted in Bulgaria (α=0.740)32 and the review studies conducted in 2004 in which the estimated range of Cronbach’s alpha for internal consistency was 0.46–0.970.33 This could be due to the consistent reliability of HAMD-17 in the current and across all the previous studies conducted using the tool.
However, the current finding on internal consistency was inconsistent with other studies conducted in Brazil (α=0.830)34 and Italy (α=0.824).10 This discrepancy might be related to cultural, population and sampling differences.
The Afaan Oromo version of HAMD-17 had an estimated area under the ROC (AUROC) of 0.966, indicating an excellent diagnostic accuracy of the tool. This estimated AUROC was comparable with the finding of the study conducted in China, which was 0.983.35 This kind of diagnostic accuracy justifies that HAMD-17 can be confidently used to screen depression indicating good discriminant power of the test in people with epilepsy who speaks Afaan Oromo.
Contrarily, the finding of this study on diagnostic accuracy was higher than the results from the Bulgaria and Lebanon studies, in which AUROC of 0.746 and 0.837 were estimated, respectively.22 32 The discrepancy might be due to the difference in the sample size variation, in which the former study used fewer sample size (n=106) compared with the latter one (n=400).
The optimal cut-off score of the HAMD-17 in people with epilepsy was found to be ≥9 in our study. At this cut-off score, the HAMD-17 demonstrated a comparable sensitivity of 92% and specificity of 91% showing high screening performance of the tool. This finding was in line with the study conducted with the Chinese version of HAMD-17 for depression in adults with epilepsy.35 This is consistent with the fact that sufficiently sensitive and specific instruments show few false negative results, and their ability to identify people who do not have a disease as a negative is adequate, respectively.
This cut-off score was higher than the findings of the studies conducted in Taiwan (cut-off poit 7)36 and Lebanon (cut-off point 7.5),22 as well as Italy (cut-off point 6).10 The possible reason for the variation might be due to the difference in cultural context.
However, this result showed a lower cut-off point than a study conducted in Poland among people with epilepsy. They used a Structured Clinical Interview (SCID-I) tool as a gold standard in which they found that the HAMD-17 polish version was valid with a cut-off score of 11 points.14 This variation can result from differences in the population’s culture and the gold standard tools (SCID-1 vs MINI).
Although this is a validation of a previously validated tool, the small sample size could compromise the power of our research. Additionally, as all people with epilepsy involved in this study were from the Jimma Medical Center, where there could be people with complex epilepsy, the result of the finding might less represent those who were attending the follow-up treatment at other primary healthcare centres. Indeed, people with intractable epilepsy could experience a more comorbid mental illness which can contribute to the enhanced validity of the tool.
Conclusions
This study found that HAMD-17 was psychometrically valid and reliable for Afaan Oromo speakers of people with epilepsy. The tool can be used by general health professionals and helps in expanding screening services given to people with epilepsy for early detection of depression. As a result, it enables workers to collaborate with other departments, such as psychiatry, for further investigations of the people.
Data availability statement
Data are available on reasonable request.
Ethics statements
Patient consent for publication
Ethics approval
Ethical clearance was acquired from Jimma University, Institutional Review Board (IRB), with a reference number of IRB000271. All the collected data were used only for this study, and the confidentiality of the participants was ensured at all stages of the study. Participants were fully informed about the objectives and methods of the study before starting the interview. All the participants were provided the written informed consent voluntarily. Finally, those who were positive for depression during data collection were linked to a psychiatry clinic for further assessment and intervention.
Acknowledgments
We would like to extend our sincere thanks to the study participants for their time and willingness to participate. We want to express our gratitude to Jimma University for its financial support in conducting this study. We also thank the Jimma University Medical Center and Psychiatry Clinic for their cooperation in allowing us to access the documents and other important information for this study.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors MY is responsible for the overall content of the paper as the guarantor. He has contributed to the design, conduct, and analyses of the research and manuscript preparation. MA, MG, and BD participated in designing the study, monitoring the data collection, and reviewing the manuscript. All authors have read and agreed on the manuscript for publication.
Funding The fund for this research was obtained from Jimma University.
Disclaimer The funder has no role in designing the study as well as, interpretation of the result, and preparation of the manuscript.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.