Article Text

Original research
Development and content validation of a financial and functional outcomes tool for diabetes-related foot disease in patients undergoing major lower limb amputation: a prospective observational study from Pakistan
  1. Nadeem Ahmed Siddiqui1,
  2. Muhammad Anees Khaliq1,
  3. Muhammad Ammar Pirzada2,
  4. Ziaur Rehman1,
  5. Fareed Shaikh1,
  6. Amna Riaz1,
  7. Sadaf Khan1
  1. 1Surgery, The Aga Khan University, Karachi, Sindh, Pakistan
  2. 2Dow International Medical College, Karachi, Sindh, Pakistan
  1. Correspondence to Dr Nadeem Ahmed Siddiqui; nads.aku{at}gmail.com

Abstract

Objective To develop and content validate a questionnaire to assess the financial and functional impact of major lower limb amputation in patients with diabetes-related foot disease.

Design Prospective observational study.

Setting This study was conducted at a tertiary care centre in Pakistan.

Participants We conducted a thorough literature review and a group interview with 10 participants, resulting in domain identification and item generation. The group included seven patients with diabetes-related foot disease who underwent major lower limb amputation and three caregivers. Subsequently, a focused group discussion was held to assess overlap and duplication among the items, and two rounds of content validation were carried out by five content and five lay experts in both English and Urdu. Question items with a Content Validity Index (CVI) score of >0.79 were retained, items with a CVI score between 0.70 and 0.79 were revised and items with a CVI score of <0.70 were excluded.

Results The initial literature review and group interview resulted in 61 items in the financial and functional domains. After the focused group discussion, the questionnaire was reduced to 37 items. Following two rounds of content validation, the English questionnaire achieved the Scale-Content Validity Index/Average (S-CVI/Ave) of 0.92 and 0.89 on relevance and clarity, respectively. Similarly, the Urdu questionnaire achieved the S-CVI-Ave of 0.92 and 0.95, respectively.

Conclusion A 37-item multidimensional questionnaire was developed and rigorously content-validated to assess the financial and functional impact of major lower limb amputation in patients with diabetes-related foot disease. The questionnaire used in this study has shown robust content validity specifically for our population.

  • diabetic foot
  • diabetic nephropathy & vascular disease
  • Health Services Accessibility

Data availability statement

Data are available on reasonable request.

http://creativecommons.org/licenses/by-nc/4.0/

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

Statistics from Altmetric.com

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

STRENGTHS AND LIMITATIONS OF THIS STUDY

  • We ensured the involvement of relevant stakeholders, including content and lay experts, giving us a comprehensive and diverse perspective.

  • The selected panel of experts possessed bilingual proficiency, allowing us to conduct content validation for both the English and Urdu versions of the questionnaire using the same panel.

  • We followed an established framework of questionnaire development and content validation.

  • The primary limitation of our study is that the interviews were conducted only with patients who received care at our tertiary care hospital, which may not be representative of the broader population.

  • An important limitation to consider with regard to content validity is the possibility of bias among experts, which could stem from the subjective nature of their feedback.

Introduction

Major lower limb amputation (MLLA) is a life-changing event for patients, as it has physical and psychological repercussions and imposes significant financial and functional challenges on the individual. These challenges adversely impact the quality of life for patients as well as their family members. Globally, the yearly incidence of major amputations resulting from diabetes-related complications was estimated to be 94.82 per 100 000 individuals from 2010 to 2020.1 Additionally, in Pakistan, the major cause of lower limb amputations is diabetes-related complications, followed by trauma.2 According to the International Diabetes Federation, in Pakistan, the occurrence of diabetes mellitus has been increasing, reaching a total prevalence of 26.7% in 2022.3 This has led to increased diabetes-related complications, including diabetes-related foot disease. The incidence of MLLA in is 94.82 per 100 000 patients with diabetes.1 A survey carried out in 25 primary care centres in Pakistan revealed that 13.9% of individuals diagnosed with type 2 diabetes were found to have diabetes-related foot disease resulting in lower limb amputation (including major and minor) in 8%–21% of patients.4

MLLA imposes a significant financial impact on both individuals and their families due to the resulting disability and these individuals are often elderly and have multiple concurrent health conditions like hypertension (65.1%), diabetes (50.6%) and coronary artery disease (45.1%).5 This aspect becomes particularly significant in a low-income and middle-income country such as Pakistan, where the financial circumstances are challenging. The absence of job security, limited access to subsidised healthcare and the physical dependency following surgery further compound the economic challenges faced by patients. The functional deterioration coupled with direct and indirect costs associated with MLLA leads to an impaired quality of life.6 A study conducted in the USA found that the mean cost associated with major amputations in patients with diabetes amounted to US$73 222.7 In a retrospective study conducted in Faisalabad, the mean direct cost of MLLA was PKR 53 720.00±12 401.24 (US$437.71±101.40).8 Although there have been studies on the direct medical and non-medical expenses associated with MLLA, the indirect costs postdischarge has not been evaluated. These indirect expenses encompass various factors such as purchasing prosthetics or crutches, loss of employment, selling an asset to compensate for the financial loss and several other associated costs. In Pakistan, the direct expenses associated with MLLA vary from one hospital to another and are highly influenced by the choices made by the patient and their family. Therefore, it is challenging to generalise the financial outcomes after MLLA. This study focuses on the direct and the indirect financial burden which patients and family must bear after MLLA.

Individuals undergoing MLLA have a profound impact on their functionality and inclusivity. A retrospective review revealed that <50% of patients are ambulatory after MLLA.9 MLLA has physical implications and it has a detrimental social impact on patients following the surgery. Patients may experience negative changes in their social interactions, including shifts in friendships, feelings of social embarrassment and, ultimately, social isolation.10 Our study seeks to create a tool to assess the functional impact that encompasses the physical aspects and includes social abilities and independent living skills. Although various studies have investigated functional outcomes following lower limb amputations, none has specifically focused on amputations resulting from diabetes-related foot disease. Diabetes and its associated health complications may affect functionality in ways different from amputation due to other causes, therefore it is crucial to evaluate functionality specifically in relation to MLLA due to diabetes-related foot disease.

Currently, there is a lack of existing tools for assessing financial and functional outcomes in individuals who have undergone MLLA as a result of diabetes-related foot disease. The aim of this study is to create, and content validate a questionnaire designed to evaluate the financial and functional consequences of MLLA in patients with diabetes-related foot disease.

Methodology

This study was conducted in the section of vascular surgery, Aga Khan University Hospital, Karachi.

Operational definitions

Major lower limb amputation

MLLA refers to the surgical removal of a portion or the entire limb, involving the cutting of bone or joint proximal to the ankle. This includes above, below, through knee amputations and hip disarticulation.11

Content experts

Experts involved in perioperative planning and care of patients undergoing MLLA. This in our study included a vascular surgeon, physiotherapist, nurse and a prosthetist. Considering the context of our study, we also included a chartered financial analyst in the expert group.

Lay experts

Patients with diabetes-related foot disease who underwent MLLA at our tertiary care centre.

Creation of the study tool

The process of creating the study tool involved two phases: (1) designing the tool through in-depth patient interviews and a literature review, followed by a focused group discussion and (2) conducting content validation of the tool by both content and lay experts.

Phase I: designing the study tool

The tool creation and content validation process we followed was based on a framework used in Iran, which focused on validating a tool designed for measuring patient-centred communication.12

The content domain identification involved conducting an in-depth group interview with seven patients with diabetes-related foot disease who had undergone MLLA and three family members. The interview was recorded, and the results were transcribed and summarised by two members of the team. The inclusion of family members in the interview was crucial, as MLLA impacts the patients and holds significant implications for their family members who play a role in their care and support. This ensured a more comprehensive representation of the impact of MLLA on both patients and their family members. The group interview revealed patients’ concerns regarding the direct expenses related to the amputation as well as the indirect financial burden associated with it. Two main content domains were identified as a result, which included postoperative functionality and financial impact on the patients.

The tool items were generated by combining findings from literature review and the outcomes obtained from the group interview. Several databases were searched for literature review including, PubMed, MEDLINE, Embase and Cochrane library using selected keywords, for example, ‘lower limb amputation’, ‘lower extremity amputation’, ‘major lower limb amputation’, ‘financial’, ‘cost’, ‘economic’, ‘functional’, ‘quality of life’, ‘diabetic foot’, ‘diabetic foot syndrome’, ‘diabetes-related foot disease’, ‘diabetic foot ulcer’, etc. The literature review revealed various studies on functional outcomes after amputation, whereas no studies were found on indirect costs associated with MLLA. As a result, multiple items were generated in each content domain by combining results from the interview with the patients and family members and the literature review.

Based on the findings from the group interview and literature review, a focused group discussion was carried out. The group consisted of two vascular surgeons, a general surgeon, two research associates, two financial advisors, a physiotherapist and the patients and their family members from the initial group interview. A questionnaire was formulated by integrating the findings and feedback gathered from the focused group discussion. Due attention was given to making sure the question items were framed in a simple language, with no ambiguity. The questionnaire was also translated in Urdu for use in our local setting, by a research associate and a surgeon, and a second research associate was involved in backward translation to English.

Phase II: content validity

The final stage that we propose for tool validation is the basis of this study. Content validation of the tool was conducted to assess clarity and relevance. An expert panel consisting of five content and five lay experts was created to give qualitative and quantitative judgements on the tool items in both English and Urdu. Care was taken to ensure that the chosen experts possessed bilingual proficiency, including a comprehensive understanding of both English and Urdu languages. Content experts included a vascular surgeon, physiotherapist, nurse, prosthetist and a chartered financial analyst. Lay experts included five patients with diabetes-related foot disease who had undergone MLLA. Written consent was taken from both the content and lay experts before the start of the content validation process. The panel members were requested to assess the clarity and relevance of each item in the questionnaire using a 4-point scale, both in English and Urdu: 4=highly relevant/clear; 3=quite relevant/clear but needs rewording; 2=somewhat relevant/clear and 1=not relevant/clear. They were also asked to give their comments and feedback on the question items. The feedback given by the experts was incorporated into the respective question items and the Item-Content Validity Index (I-CVI) score was calculated for each question. I-CVI was calculated as the number of experts giving a rating of ‘highly relevant/clear’ or ‘quite relevant/clear but needs rewording’ for each item divided by the total number of experts. Question items with an I-CVI score of >0.79 were retained, items with an I-CVI score between 0.70 and 0.79 were revised and items with an I-CVI score of <0.70 were excluded.13 Modified kappa statistic was used to provide information about the degree of agreement beyond chance.14 Kappa values were categorised as follows: values >0.74 were considered excellent, values between 0.60 and 0.74 were classified as good and values ranging from 0.40 to 0.59 were considered fair.15 As a result, certain questions were rephrased, leading to the completion of the final questionnaire. The Scale-Content Validity Index/Average (S-CVI/Ave) for the entire tool was calculated by taking the sum of the I-CVIs divided by the total number of items.12

Figure 1 shows a flow diagram of the development and content validation process of the questionnaire. After conducting a thorough literature review and interviewing both patients and the family members, a total of 32 items in the financial and 29 items in the functional domain were identified. A focused group discussion was conducted to thoroughly examine these items for overlap and duplication. Finally, a questionnaire consisting of 19 financial items and 18 functional items were generated (see online supplemental file 1 for English version and online supplemental file 2 for Urdu version).

Figure 1

A flow diagram of the development and content validation process.

Patient and public involvement

Patients were involved as study participants and lay experts for creation of the study questionnaire and content validation, respectively. However, content experts were only involved in the process of content validation. Inputs from both the content and lay experts were used to generate the final questionnaire. All included patients as study participants and lay experts had undergone MLLA due to diabetes-related foot disease within the past 6 months.

For a group interview, potential participants were contacted by phone call, where the study objectives were explained, and invitations to join the study were extended. Seven patients and three family members agreed to participate and were included for the purpose of domain identification and item generation. A scheduled group interview followed, where verbal consent was obtained from all participants by the primary investigator before commencement. The interview sessions were recorded, and the subsequent transcription and summarisation of results were performed by two team members.

For content validation, patients were approached by a member of the research team in the hospital waiting area before they consulted with the vascular surgeon. After explaining the purpose of the study, patients were invited to participate. If they agreed, an appointment was made after their consultation in the clinic. Written consent was taken from the patients before asking to assess the questionnaire. Patients were then asked to assess the relevance and clarity of the questionnaire items on a hard copy.

Content experts were contacted through a telephone call, during which they were provided with an overview of the study’s objective and invited to participate in the study. On their agreement, a hard copy of the consent form and questionnaire was sent to them for assessing relevance and clarity. After completing the assessment, the content experts contacted a team member to submit both the consent form and the questionnaire.

The process was repeated for the second round of content validation for both the content and lay experts.

Results

Tables 1 and 2 presents the results of the CVI scores based on relevance and clarity for both versions of the prefinal questionnaire. The results of our study indicate that most of the items performed well. Items with a CVI score exceeding 0.78 for both clarity and relevance were retained. None of the items were excluded from the prefinal questionnaire, as each item scored 0.7 or higher on both indices. The S-CVI/Ave score for the prefinal questionnaire in English based on relevance and clarity was 0.91 and 0.87, respectively. Similarly, the S-CVI/Ave score for the prefinal questionnaire in Urdu was 0.92 and 0.94, respectively. A total of four items from the English prefinal questionnaire and one item from the Urdu prefinal questionnaire scored within the range of 0.70–0.79 in either relevance or clarity. These items were revised based on the recommendations of the expert panel to improve their relevance and clarity. Item 5 in the financial category, for example, was revised from “Did you lose your job?” to “Did you lose your job after the surgery?” Similarly, item 11 was revised from “Are you involved in any recreational activity?” to “Are you involved in any recreational activity (eg, cricket, football, eating out, shopping, picnics, etc)?” The selected items were subjected to a subsequent round of content validation by the same panel of experts. The results for these items are given in table 3. The results for the revised items show that all the items performed well in both the indices.

Table 1

CVI scores for relevance and clarity of financial and functional items of the English questionnaire

Table 2

CVI scores for relevance and clarity of financial and functional items of the translated Urdu questionnaire

Table 3

CVI scores for relevance and clarity for the modified items

The five items that were modified and the remaining original question items (part of the prefinal questionnaire) were merged to form a final questionnaire. A total S-CVI/Ave score was calculated for the final questionnaire based on both relevance and clarity. The total S-CVI/Ave score for the final English questionnaire based on relevance and clarity was 0.92 (from 0.91) and 0.89 (from 0.87), respectively. Similarly, for the Urdu questionnaire the S-CVI/Ave score for relevance stayed the same (0.92) as no items were modified, whereas for clarity the S-CVI/Ave was 0.95 (from 0.94).

Discussion

This is the first of its kind study to rigorously develop, and content validate a questionnaire that can assess financial and functional impact in patients with diabetes-related foot disease undergoing MLLA. After interviewing the patients who underwent MLLA, these two domains were frequently observed to be adversely impacted. With due consideration to the financial, social and cultural dimensions of our target population, we developed a comprehensive questionnaire, incorporating question items specifically tailored to our cohort.

We followed the protocol outlined by Zamanzadeh et al in the development and content validation of the questionnaire.12 Items were generated by reviewing literature and interviewing the patients and their family members. We formed an expert panel of 10 members to avoid overestimation of the content validity. The 10 experts included 5 content and 5 lay experts. Five lay experts were included to make sure that the target population was being represented.16 The revised questions underwent an additional round of content validation conducted by the same panel of experts to ensure all items were appropriately phrased and accurately represented the intended constructs they were designed to measure. This methodology of questionnaire development and validation is adopted and reported by many studies.17–20 Different strategies of questionnaire content validation have been reported by other studies,21 22 but we find this study more rigorous and feasible for our study population.

MLLA has a significant impact on patient’s functional status and leads to adverse financial outcomes for the patients and their family members. Moreover, it can adversely impact the employment rates postamputation from as low as 48% to as high as 89% as seen in the previous studies on patients undergoing MLLA due to traumatic injury.23 Usually, people end up doing less physically demanding work or need changes in their job tasks instead of going back to their previous physically demanding jobs.24 Our questionnaire includes functional domains that encompass items related to returning to work and factors contributing to the decision of not returning to work and changes in job roles. The functional category of the questionnaire also encompassed items pertaining to various aspects of patients’ postamputation quality of life, such as activities of daily living, social functioning, religious activities, engagement in recreational activities and the status of their spousal relationship.

The financial category of the questionnaire has items that assess the impact on patients and their family members specifically applicable to our population. Within our specific context, the economic situation is challenging, with a substantial portion (37.2% as of 2023) of the population residing near the poverty line.25 Consequently, procedures like MLLA can have profound and devastating effects on both patients and their family members. In a resource-limited country like Pakistan, direct and indirect costs associated with MLLA can have a significant economic burden. Hence, our questionnaire can serve as a foundational tool for identifying and addressing the financial impact on patients and their family members, aiming to prevent and mitigate its adverse effects.

Several quality-of-life assessment tools are available including SF-36 and SF-12 which focus on overall patient well-being.26 27 VascuQOL evaluates quality of life in peripheral arterial disease patients comprehensively.28 However, these tools primarily focus on a wide demographic and evaluate overall patient well-being, encompassing physical, mental and emotional aspects. In contrast, our questionnaire is specifically designed for patients with diabetes-related foot disease who have undergone MLLA. It encompasses various quality of life measures, including social, physical, mental and emotional well-being, as well as the financial well-being of both patients and their family members.

This questionnaire is intended for clinicians to administer at least 6 months post-MLLA. This timeframe ensures sufficient time has passed to evaluate the financial and functional impacts on patients with diabetes-related foot conditions. In the future, our goal is to implement this survey on a broad scale, encompassing both private and public sectors, to gauge the prevalence of these challenges in our population. This will serve as the basis for providing guidance to clinicians on how to address these challenges effectively.

Our study has multiple strengths. This study is the first of its kind within our population to create a questionnaire that is specifically relevant and applicable to our unique circumstances. It aimed to comprehensively evaluate the indirect expenses associated with MLLA. We followed an established framework of questionnaire development and content validation proposed by Zamanzadeh et al.12 Additionally, we ensured the involvement of all relevant stakeholders, including content and lay experts giving us a comprehensive and diverse perspective for questionnaire development and validation. The selected panel of experts possessed bilingual proficiency, allowing us to conduct content validation for both the English and Urdu versions of the questionnaire using the same panel.

Our study is subject to certain limitations, primarily stemming from the fact that our interviews were restricted to patients who exclusively sought care at our tertiary care hospital. Consequently, the viewpoints expressed may not be fully representative of the broader population under consideration. Moreover, it is important to acknowledge an important limitation regarding content validity, which is the potential for bias that may exist among experts due to the subjective nature of their feedback. Nonetheless, this study can serve as a basis for future studies to conduct a pilot study to further perform psychometric analysis.

Conclusion

A 37-item multidimensional questionnaire was developed and rigorously content validated to assess the financial and functional impact of MLLA in patients with diabetes-related foot disease. The questionnaire used in this study has shown robust content validity specifically for our population. Future studies can be done to use this questionnaire to conduct a pilot study.

Data availability statement

Data are available on reasonable request.

Ethics statements

Patient consent for publication

Ethics approval

This study was approved by the Ethics Review Committee (ERC) of Aga Khan University Hospital (ERC approval number 2022-7192-22309). Participants gave informed consent to participate in the study before taking part. The consent was taken by the primary investigator.

References

Supplementary materials

  • Supplementary Data

    This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.

Footnotes

  • Twitter @AmmarPirzada

  • Contributors NAS was involved in study concept, conduct, data collection, manuscript writing and proof reading. MAK, MAP, ZR, FS and AR were involved in data collection, manuscript writing and proof reading. SK was involved in study concept, design and proof reading. NAS accepts full responsibility for the work and/or the conduct of the study, had access to the data, and controlled the decision to publish.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Patient and public involvement Patients and/or the public were involved in the design, or conduct, or reporting, or dissemination plans of this research. Refer to the 'Methods' section for further details.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.