You are here

  • Home
  • Archive
  • Volume 7, Issue 2
  • American Orthopaedic Foot and Ankle Society (AOFAS) Ankle-Hindfoot Score: a study protocol for the translation and validation of the Dutch language version

Article Text

Article menu

Download PDFPDF

XML
Surgery
Protocol
American Orthopaedic Foot and Ankle Society (AOFAS) Ankle-Hindfoot Score: a study protocol for the translation and validation of the Dutch language version
  1. Esther M M Van Lieshout1,
  2. A Siebe De Boer1,
  3. Duncan E Meuffels2,
  4. P Ted Den Hoed3,
  5. Cornelis H Van der Vlies4,
  6. Wim E Tuinebreijer1,
  7. Michael H J Verhofstad1
  1. 1Trauma Research Unit Department of Surgery, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
  2. 2Department of Orthopaedic Surgery, Erasmus MC, University Medical Center Rotterdam, Rotterdam, The Netherlands
  3. 3Department of Surgery, Ikazia Hospital, Rotterdam, The Netherlands
  4. 4Department of Surgery, Maasstad Hospital, Rotterdam, The Netherlands
  1. Correspondence to Dr Esther MM Van Lieshout; e.vanlieshout{at}erasmusmc.nl

Abstract

Introduction The American Orthopaedic Foot and Ankle Society (AOFAS) Ankle-Hindfoot Score is among the most commonly used instruments for measuring the outcome of treatment in patients who sustained a complex ankle or hindfoot injury. It combines a clinician-reported and a patient-reported part. A valid Dutch version of this instrument is currently not available. Such a translated and validated instrument would allow objective comparison across hospitals or between patient groups, and with shown validity and reliability it may become a quality of care indicator in future. The main aims of this study are to translate and culturally adapt the AOFAS Ankle-Hindfoot Score questionnaire into Dutch according to international guidelines, and to evaluate the measurement properties of the AOFAS Ankle-Hindfoot Score-Dutch language version (DLV) in patients with a unilateral ankle or hindfoot fracture.

Methods and analysis The design of the study will be a multicentre prospective observational study (case series) in patients who presented to the emergency department with a unilateral ankle or hindfoot fracture or (fracture) dislocation. A research physician or research assistant will complete the AOFAS Ankle-Hindfoot Score-DLV based on interview for the subjective part and a physical examination for the objective part. In addition, patients will be asked to complete the Foot Function Index (FFI) and the Short Form-36 (SF-36). Descriptive statistics (including floor and ceiling effects), internal consistency, construct validity, reproducibility (ie, test–retest reliability, agreement and smallest detectable change) and responsiveness will be assessed for the AOFAS DLV.

Ethics and dissemination This study has been exempted by the Medical Research Ethics Committee (MREC) Erasmus MC (Rotterdam, the Netherlands). Each participant will provide written consent to participate and remain anonymised during the study. The results of the study are planned to be published in an international, peer-reviewed journal.

Trial registration number NTR5613. pre-result.

  • Dislocation
  • Fracture
  • Reliability
  • Responsiveness
  • Validity

This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2016-012884

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Strengths and limitations of this study

    • This study involves translation and validation of the American Orthopaedic Foot and Ankle Society (AOFAS) Ankle-Hindfoot Score into Dutch.

    • It is a prospective multicentre observational study with a strong methodological design.

    • Statistical analyses will comply with the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines.

    • The study is limited to adults (aged 18 years or older) who have adequate comprehension of the Dutch language.

    • Although the study will be mostly relevant for the Dutch-speaking regions, it is also informative for other regions.

    Introduction

    Complex foot and ankle injuries cause a usually temporary loss of function and quality of life. Patient-reported outcome measures (PROMs) are essential in clinical practice and clinical research; they enable a detailed evaluation of (functional) outcome or quality of life after (non-)operative treatment of musculoskeletal (traumatic) injuries from a patient's perspective. Generic instruments such as quality of life questionnaires allow comparison across populations with different injuries or medical conditions. Region-specific instruments, on the other hand, may give more detailed insight into the disabilities, pain and problems caused by a specific injury. Some instruments are solely PROMs, and others combine a patient-reported with a physician-reported part. Numerous generic and region-specific instruments are available.1–6

    A frequently used instrument for assessing outcome after ankle and hindfoot injuries is the American Orthopaedic Foot and Ankle Society (AOFAS) Ankle-Hindfoot Score. This clinical rating system, developed by Kitaoka et al,7 combines subjective scores of pain and function provided by the patient with objective scores based on the surgeon's physical examination of the patient (to assess sagittal motion, hindfoot motion, ankle–hindfoot stability and alignment of the ankle–hindfoot). The scale includes nine items that can be divided into three subscales (pain, function and alignment). Pain consists of one item with a maximal score of 40 points, indicating no pain. Function consists of seven items with a maximal score of 50 points, indicating full function. Alignment consists of one item with a maximal score of 10 points, indicating good alignment. The maximal score is 100 points, indicating no symptoms or impairments. In the original publication, the AOFAS Ankle-Hindfoot Score was described to be used for ankle replacement, ankle arthrodesis, ankle instability operations, subtalar arthrodesis, subtalar instability operations, talonavicular arthrodesis, calcaneocuboid arthrodesis, calcaneal osteotomy, calcaneus fracture, talus fracture and ankle fractures.7

    Evidence that the AOFAS Ankle-Hindfoot Score (as a complete scale) is valid in its original version is limited.7–9 Poor-to-moderate correlation of the AOFAS scores to the Short Form-36 Health Survey (SF-36) subscales may also suggest poor construct validity.10 Adequate responsiveness has been shown.8 ,9 The physician-reported part of the scale has been shown to be valid and reliable.11 Westphal et al12 showed that correlations between SF-36 and the AOFAS Ankle-Hindfoot Score were strong regarding function and pain subscales, but moderate for all other subscales. Previous studies involved a wide spectrum of diagnoses, such as general ankle–hindfoot symptoms,9 pending ankle or foot surgery,11 surgically treated calcaneal fractures12 and end-stage ankle arthritis.8 Some of these studies have included mixed populations.

    Despite some favourable results, there is also criticism of the use of the AOFAS Clinical Rating Systems, which includes the AOFAS Ankle-Hindfoot Score.13 Criticism, which includes the limited number of answers per item as well as linguistic issues, may negatively affect reliability and validity, and makes it more prone to ceiling effects.13 ,14 Despite these concerns, the AOFAS Ankle-Hindfoot Score remains among the most commonly used instruments, especially for patients with hindfoot fractures. It is an especially interesting instrument because it asks for hindfoot-specific symptoms or deviations, which are not included in other lower extremity-specific instruments.

    Currently, a validated Dutch translation of the AOFAS Ankle-Hindfoot Score is not available. Therefore, the aim of the first part of the study is to translate and culturally adapt the AOFAS Ankle-Hindfoot Score questionnaire into Dutch. The aim of the second part is to evaluate the measurement properties of the AOFAS Ankle-Hindfoot Score-Dutch language version (DLV) in patients who sustained a unilateral ankle or hindfoot fracture or (fracture) dislocation by assessing descriptive statistics (including floor and ceiling effects), internal consistency, construct validity, reproducibility (ie, test–retest reliability, agreement and smallest detectable change (SDC)) and responsiveness. Measurement properties will be calculated separately for the ankle and hindfoot.

    Methods and analysis

    Study design

    This study (protocol V.1.0, date 24 March 2014) will follow a multicentre prospective observational study design (ie, case series). Since the research physician and patients will complete questionnaires starting at variable time points during treatment, this study will have a prospective study design with retrospective data collection with regard to the injury and treatment. Three hospitals in Rotterdam (the Netherlands) will participate: Erasmus MC, University Medical Center Rotterdam, Ikazia Hospital and Maasstad Hospital. The study is registered at the Netherlands Trial Register (NTR5613), registration date 5 January 2016.

    Recruitment and consent

    All consecutive patients meeting the eligibility criteria (and none of the exclusion criteria) will be included. Participation in this study will not have any influence on treatment. Prior to their outpatient department visit, eligible patients will be invited to participate. Verbal and written information will be given by the principal investigator, research physician or a research assistant. Written materials will include an information letter, informed consent form and return envelope. A reminder will be sent to those patients who did not respond within 2 weeks, in order to ensure a high response rate. If no response is received within 3 weeks, the patient will be contacted by telephone.

    In order to reduce bias as much as possible, a research physician (MD with clinical experience) or research assistant (with a BSc in medicine) will perform the physical examination that is part of the physician-reported part of the AOFAS Ankle-Hindfoot Score-DLV using a standardised protocol. Both assessors received elaborate training on the administration and physical examination of the AOFAS Ankle-Hindfoot Score by an experienced trauma surgeon.

    Study population

    All adult patients who visited the emergency department of any of the participating hospitals and were diagnosed with a unilateral ankle or hindfoot fracture or (fracture) dislocation will be considered eligible for inclusion. Measurement properties will be assessed separately for the ankle and the hindfoot subgroups. Patients will be identified from hospital records based on their International Coding of Diseases, 10th revision (ICD-10) code or Diagnosis Related Group (DRG; in Dutch, DBC) code.

    Three subgroups of patients will be enrolled. In group 1 (test of prefinal version) the prefinal version of the AOFAS Ankle-Hindfoot Score-DLV will be completed. In group 2 (responsiveness) and group 3 (test–retest), the final version of the Dutch AOFAS Ankle-Hindfoot-DLV questionnaire will be completed on two occasions, with 5–6 months (group 2) or 2–3 weeks (group 3) in between.

    In order to be eligible to participate in this part of the study, a patient must meet all of the following criteria:

    1. Patients with a unilateral ankle or hindfoot fracture or (fracture) dislocation (ie, ankle fracture, calcaneal fracture, talar fracture, subtalar dislocation, tibiotalar dislocation or Chopart's fracture dislocation);

    2. Age 18 years or older;

    3. Group 2 only: treatment started between 6 weeks and 3 months (ankle) or between 3 and 6 months (hindfoot) prior to the start of the study;

    4. Group 3 only: treatment has started between 7 and 9 months (ankle) or between 6 and 24 months (hindfoot) prior to the start of the study;

    5. Provision of informed consent by patient.

    A potential participant who meets any of the following criteria will be excluded from participation in this study:

    1. Patient with multiple trauma (only if functional recovery of additional injuries was not achieved at the time of enrolment, as that most likely affects the outcome scores);

    2. Pathological fracture;

    3. Severe physical comorbidity (ie, American Society of Anesthesiologists (ASA) ≥3);

    4. Patient was non-ambulatory prior to the injury (ie, bed-bound or wheelchair user);

    5. Insufficient comprehension of the Dutch language to understand and complete the questionnaires

    6. Patient with expected problems of maintaining follow-up (eg, no fixed address)

    For testing the prefinal version of the Dutch AOFAS Ankle-Hindfoot Score-DLV (group 1), only exclusion criteria 5 and 6 will apply.

    Patients are allowed to participate in groups 2 and 3, and if so, the second questionnaire for responsiveness will also be used as the first questionnaire for test–retest reliability. Table 1 shows a summary of the injuries, identifying codes and measurement times of this study.

    View this table:
    Table 1

    Overview of injuries, identifying codes and measurement times

    Outcome measures

    The measurement properties of the AOFAS Ankle-Hindfoot Score-DLV will be evaluated in this validation study. The following parameters will be determined:

    • Construct validity;

    • Reliability/internal consistency;

    • Reproducibility: test–retest reliability, agreement and smallest detectable change;

    • Floor and ceiling effects;

    • Responsiveness.

    In addition to the outcome variables aforementioned, the following data will be collected from the patients' medical files:

    • Intrinsic variables (baseline data): age, gender and dominant side.

    • Injury-related variables: affected side, trauma mechanism, type of injury.

    • Intervention-related and outcome-related variables: type of treatment (operative or non-operative), time between injury and start of treatment, achievement of anatomic restoration as judged from an X-ray or CT scan (ie, <2 mm articular step-off or gap).

    Study procedures

    The study will be divided into two stages. First, the American (original) version of the AOFAS Hindfoot-Ankle Score will be translated into Dutch according to a standardised procedure.15 Second, the translated version will be tested for measurement properties in a prospective study.

    Step 1: translation of the questionnaire

    The translation and cultural adaptation of the AOFAS Ankle-Hindfoot Score questionnaire will be done according to the guideline for Cross Cultural Adaptation of Self-Report Measures by Beaton et al.15 This guideline is based on the review of Guillemin et al16 and is the official guideline of the American Academy of Orthopaedic Surgeons. The guideline consists of five stages: (1) translation; (2) synthesis; (3) back translation; (4) evaluation by a team of experts and (5) tests.

    In stage 1, the English version of the questionnaire will be translated into Dutch independently by two Dutch native speakers who are fluent in English. One person will have knowledge of medicine and the questionnaire, while the other will not necessarily.

    In stage 2, both translations will be combined by the two translators and a team of experts; this team will consist of at least two independent observers. The synthesis process will be carefully documented in a written report. Differences will be resolved by consensus.

    In stage 3, two persons will independently translate the synthesised Dutch questionnaire back into English. Both translators will be bilingual native English speakers. Neither translator will receive any background information on the study or the questionnaire. They will have no medical background, will be blind to the original version of the questionnaire and will not be aware or informed about the concepts explored in it. With this back-translation process, the content validity of the questionnaire is checked in order to make sure that the translated version is reflecting the same item content as the original version. Unclear wording in the translated version can be discovered in this stage.

    In stage 4, the investigator, the translators and the same team of experts will review the two back-translations. Equivalence between the original and Dutch versions of the questionnaire shall be reached in four areas: semantic equivalence (ensuring that the words mean the same thing), idiomatic equivalence (ensuring that colloquialisms or idioms are formulated in equivalent expressions), experiential equivalence (ensuring that each item captures the experience of daily life in the target culture) and conceptual equivalence (ensuring that words hold the same conceptual meaning). Discrepancies will be resolved by consensus. This stage will result in the prefinal Dutch versions of the questionnaire.

    In stage 5, these prefinal Dutch version will be tested in a group of 20 patients (group 1) presenting themselves with various foot/ankle problems to the outpatient clinic of one of the participating hospitals. These patients will be asked if they understand the questions and if they are able to provide answers to the questions. If all patients report that this is the case and if there are no ambiguities, no further changes to the questionnaires will be necessary; at that point, the translated questionnaire will be considered final. The measurement properties of this version will be assessed in Dutch patients as described below.

    Step 2: determining measurement properties of the AOFAS Ankle-Hindfoot Score-DLV

    Patient groups 2 and 3 will be used for this evaluation.

    • Group 2 (responsiveness) will consist of patients who were (surgically) treated at a participating hospital, between 6 weeks and 3 months earlier (ankle) or between 3 and 6 months earlier (hindfoot).

    • Group 3 (test–retest) will consist of patients who were (surgically) treated at a participating hospital, between 7 and 9 months earlier (ankle) or between 6 and 24 months earlier (hindfoot).

    In groups 2 and 3, three questionnaires will be completed during the patient's outpatient department visit: the AOFAS Ankle-Hindfoot Score-DLV, the Foot Function Index (FFI-DLV)2 and the SF-36-DLV.17 These instruments were chosen since they were also used for the validation of the original language version.8 The research physician or research assistant will complete the AOFAS Ankle-Hindfoot Score-DLV during the outpatient department visit. If a patient is unable or unwilling to come to the hospital, a home visit may be planned.

    The FFI measures the effect of foot pathology on function in terms of pain and disability. The FFI consists of 23 items divided into three subscales: limitation, pain and disability. The items are scored on a 10-point Likert scale. For each subscale, the raw score is transformed to a 100-point score; the higher the score, the more the limitation/pain/disability that is present. The total score on the FFI is the mean of the subscale scores.2 Adequate internal consistency, reproducibility and reliability as well as strong correlation with SF-36 have been reported for patients with traumatic foot disorders in some languages.2 ,18 ,19 The FFI-DLV will be used.2

    The SF-36 is a generic health status questionnaire that gives an indication of health-related quality of life.20–27 The SF-36 consists of 36 items (questions) and provides scores on eight dimensions (subscales): physical functioning (PF), role limitations due to physical health problems (RP), bodily pain (BP), general health perceptions (GH), vitality (VT), social functioning (SF), role limitations due to emotional problems (RE) and general mental health (MH). These eight domains are combined into a physical component summary (PCS) and a mental component summary (MCS). The raw score on each subscale is transferred to a 100-point scale, with a higher score indicating better quality of life. These scores will be converted to a norm-based score and compared with the norms for the general population of the USA (1998), in which each scale was scored to have the same average (50 points) and the same SD (10 points). Dutch norms are available, but will not be used. The Dutch norms were calculated using a smaller sample size than the American study. Moreover, most published studies have used the American norms. On a study population level, the means and median values were similar when using the Dutch or American norms, but variance was larger when using the Dutch norms than when using the US norms.28 The SF-36 is the most widely evaluated PROM for assessing general health.29 It is reliable and easy to complete. A validated Dutch version will be used.17

    In order to determine whether the AOFAS Ankle-Hindfoot Score-DLV is able to detect clinical change over time, patients in group 2 will be asked to complete all questionnaires again after 5–6 months after completing them the first time. A research physician or research assistant will complete the AOFAS Ankle-Hindfoot Score-DLV. For responsiveness, this time interval should be sufficiently long enough for clinical improvement to occur. We consider a time interval of 5–6 months to be appropriate for all three groups of injuries.

    In order to determine the reproducibility (ie, test–retest reliability) of the AOFAS Ankle-Hindfoot Score-DLV, all questionnaires will be completed again at 2–3 weeks after completing them the first time (group 3). For test–retest reliability, this time interval needs to be sufficiently short to support the assumption that the patient remains stable and sufficiently long to prevent recall. We consider a time interval of 2–3 weeks to be appropriate. Patients are asked about the presence or absence of change between the two questionnaire administrations. They were asked to complete a transition item (anchor question) evaluating their perception of change in the general condition of their affected ankle. The question was: How would you judge the condition of your ankle, compared with the last time you completed this questionnaire? Patients were given the answer options ‘better’, ‘no change’ or ‘worse’. Patients reporting a change (either improvement or deterioration) will be excluded from the analysis. Patients who replied ‘no change’ were considered stable between the two measurements.

    Sample size calculation

    The prefinal Dutch version of the instrument will be tested in a group of 20 patients (group 1) presenting themselves with various foot/ankle problems to the outpatient clinic of the Erasmus MC (Rotterdam), Ikazia Hospital (Rotterdam) or Maasstad Hospital (Rotterdam).

    For groups 2 and 3, recruitment of the ankle and the hindfoot injury subgroups will continue until complete follow-up is ensured for 100 patients. The minimum number of patients needed for determining measurement properties of a PROM depends on the property evaluated. Validity can only be rated positive if at least 75% of the results are in correspondence with prespecified hypotheses, in (sub)groups of at least 50 patients.30 For calculating the SDC as well as for the assessment of the agreement parameters (reproducibility), a sample size of at least 50 patients is generally considered adequate.30 ,31 The (absence of) floor and ceiling effects also requires a sample size of at least 50 patients. In order to perform a factor analysis (to determine if the AOFAS Ankle-Hindfoot Score-DLV consists of multiple subscales), however, 4–10 patients for each item are advised with a minimum of 100 patients.30 ,32 The sample size needed applies both to patients with ankle injuries and hindfoot injuries.

    Statistical analysis

    Data will be entered and encoded into an OpenClinical database. A random sample of entered data will be checked by an independent data monitoring committee. Only the research team, the Medical Research Ethics Committee (MREC) and the health inspection will have legal access to the data.

    All statistical analyses will be performed with the Statistical Package for Social Sciences (SPSS, V.21 or higher) and will be reported following the STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) and the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) guidelines. Descriptive statistics will be used in order to describe the main characteristics of the study participants and the questionnaire scores at the different time points. Data for patients with ankle or hindfoot injuries will be evaluated as two separate groups.

    Since the raw data for individual items will be analysed, missing values will not be imputed. Normality of continuous data will be tested with the Shapiro-Wilk test. Descriptive analysis will be performed; continuous data will be reported as mean±SD (parametric) or median with percentiles (non-parametric) and categorical data as numbers with percentages.

    In order to evaluate if a representative sample participated in this study, the age, gender and injury location of responders will be compared with that of the non-participants. The categorical variables gender and injury location will be assessed using a χ2 test. Age will be compared using Student's t-test (parametric data) or the Mann-Whitney U test (parametric data).

    Construct validity

    Validity is the degree to which a patient-reported outcome instrument measures the construct it is supposed to measure. Since there is no gold standard in the current study, the validity of the AOFAS Ankle-Hindfoot Score-DLV will be expressed in terms of the construct validity. Construct validity refers to the extent to which scores on a specific questionnaire relate to other measures in a way that is in agreement with prior theoretically derived hypotheses concerning the concepts that are being measured.30 In order to evaluate the construct validity of the AOFAS Ankle-Hindfoot Score-DLV, we will formulate a set of hypotheses about the expected magnitude and direction of relationships between the AOFAS (sub)scores and the FFI and the SF-36 (sub)scores. Pearson's product-moment correlation coefficients (parametric data) or Spearman's r (rank correlation) coefficients (non-parametric correlation) will be calculated in order to assess construct validity. Correlation coefficients above 0.6, between 0.6 and 0.3 and <0.3 will be considered high, moderate and low correlations, respectively.33 The AOFAS Ankle-Hindfoot Score is expected to have a high correlation with pain and function (sub)scales (ie, FFI total score and all three subscales, SF-36 PF, RP, BP and PCS), a moderate correlation with the SF-36 VT, SF and RE subscales, and a low correlation with SF-36 GH, MH and MCS. Construct validity will be given a positive rating if at least 75% of the results are in accordance with predefined hypotheses in a (sub)sample of at least 50 patients.30

    Reliability/internal consistency

    Reliability is defined as the degree to which the measurement is free from measurement error.34 Three elements of reliability will be determined: internal consistency, reproducibility and measurement error.

    Internal consistency is defined as the extent to which items in a (sub)scale are intercorrelated, thus measuring the same construct.30 The correlation between items on a (sub)scale will be evaluated by calculating Cronbach's α for every (sub)scale. Since future use of the AOFAS instrument will be at a group level, internal consistency is considered sufficient if the value for Cronbach's α is between 0.70 and 0.95, provided that the scale is unidimensional.30 ,35 ,36 If necessary, confirmatory or exploratory factor analysis will be performed, as applicable.

    Reproducibility

    Reproducibility concerns the degree to which repeated measurements in stable persons (test–retest) provide similar answers.30 Reproducibility is suggested to consist of two parts: reliability and agreement.37 ,38 The data of group 3 will be used; they will complete all questionnaires twice, with 2–3 weeks in between. Only data for patients reporting ‘no change’ on the transition item are included as they were considered to be stable between the measurements.

    Reliability concerns the degree to which patients can be distinguished from each other, despite measurement error.30 ,39 Evaluation of the test–retest reliability of the AOFAS Ankle-Hindfoot Score-DLV will be performed by calculating the intraclass correlation coefficient (ICCagreement) with a corresponding 95% CI. An ICC two-way random-effects model, type absolute agreement (ICC(2,1)), will be used.40 Reliability will be given a positive rating when the ICC is at least 0.70 in a sample size of at least 50 patients.30

    Agreement concerns the absolute measurement error, that is, how close the scores on repeated measures are expressed in the unit of the measurement scale at issue.30 The degree of absolute agreement of the AOFAS Ankle-Hindfoot Score-DLV will be expressed as the SE of measurement (SEMagreement). This SEM equals the square root of the error variance of an analysis of variance analysis, including the systematic differences (SEM=√(variancepatient+varianceresidual).30 ,41 ,42

    Based on the SEM, the SDC will be calculated using the formula: SDC=1.96×√2×SEM.30 The SDC reflects the smallest within-person change in a score that, with p<0.05, can be interpreted as a ‘real’ change, above measurement error, in one individual (SDCind).30 ,43 ,44 The SDC measurable in a group of people (SDCgroup) will be calculated by dividing the SDCind by √n.44 ,45 Finally, the reliable change index will be calculated, representing the SDC as a percentage of the maximum obtainable score.

    The degree of absolute agreement of the AOFAS Ankle-Hindfoot Score-DLV will also be determined with a Bland and Altman analysis.46 The limits of agreement equal the mean change in scores of repeated measurements (meanchange)±1.96×SD of these changes (SDchange).30 Zero falling outside this interval indicates a bias in the measurements.

    Floor and ceiling effects

    The validity, reliability and responsiveness of a questionnaire may be jeopardised if floor or ceiling effects are present. It is then likely that extreme items are missing in the lower or upper ends of the questionnaire. As a consequence, respondents with the lowest or highest possible score cannot be distinguished from each other (indicating limited reliability) and changes in these patients cannot be measured (indicating limited responsiveness).30 Floor and ceiling effects will be determined by calculating the number of individuals who obtained the lowest (0 points; floor) or highest (100 points; ceiling) scores possible and will be considered present if more than 15% of the respondents achieved the lowest or highest score in a sample size of at least 50 patients.30 ,47 Floor and ceiling effects will be determined separately for the different time points.

    Responsiveness

    Responsiveness is defined as the ability of a questionnaire to detect clinically important changes over time, even if these changes are small.30 ,48 The data of group 2 will be used; they will complete all questionnaires twice, with 5–6 months in between.

    The effect size (ES) and standardised response mean (SRM) of the (sub)scales of the AOFAS Ankle-Hindfoot Score-DLV will be determined as measures of the magnitude of change over time. The ES will be calculated by dividing the mean change in score between the two time points by the SD of the first measurement.49 The SRM will be calculated by dividing the mean change in score between two time points by the SD of this change.49 These effect estimates will be interpreted according to Cohen: an SRM of 0.2–0.4 is considered a small effect, 0.5–0.7 a moderate effect, and 0.8 or higher a large effect.50

    Responsiveness can be considered to be a measure of longitudinal validity. In analogy to construct validity, this longitudinal validity will be assessed by testing predefined hypotheses about expected correlations between changes in AOFAS Ankle-Hindfoot Score-DLV (sub)scales versus changes in FFI and SF-36 (sub)scales.30 Change scores of the AOFAS Ankle-Hindfoot Score are expected to have a moderate correlation with changes in the FFI (sub)scales, SF-36 PF, RP, BP, VT, SF, RE and PCS. A low correlation is expected with changes in the SF-36 GH, MH and MCS.

    Ethics and dissemination

    This study will be conducted according to the principles of the Declaration of Helsinki (64th World Medical Association General Assembly, Fortaleza, Brazil, October 2013). This study has been exempted by the MREC Erasmus MC (Rotterdam, the Netherlands). This MREC acts as central ethics committee for this trial (reference number MEC-2014-215). Approval has been obtained from the local hospital boards in all participating centres. Following a review of the protocol, the MREC concluded that this study is not subject to the Medical Research Involving Human Subjects Act (WMO). They concluded that the study is a medical/scientific research, but no patients are subjected to procedures or are required to follow rules of behaviour. Consequently, the statutory obligation to provide insurance for participants participating in medical research (article 7 of the WMO) was also waived. Any important changes in the protocol will be submitted to the accredited MREC. The results of the study are planned to be published in an international peer-reviewed journal. Results of the ankle and hindfoot injury subgroups will be published separately.

    Discussion

    Modern studies that evaluate treatment efficacy are expected to also take into account the treatment outcome from a patient's perspective. Clinical measures such as mortality, radiographic healing, and rates of complications, reoperation and readmission are relevant; however, they do not reflect to what extent a patient is able to function in daily living. For that purpose, PROMs and mixed instruments, which combine a patient-reported and a physician-reported part, have been developed. There is a great need for valid instruments in different languages.

    The AOFAS Ankle-Hindfoot Score is commonly used in patients with an ankle or hindfoot injury. This instrument combines functional outcome and pain, which are both critical for patients. The AOFAS Ankle-Hindfoot Score is only valid if the score truly reflects function and pain. Completing the questionnaire in duplicate should result in the same score, and during recovery, the change in score should reflect change in the functional status of the patient. Both elements of validity of the instrument are determined as part of this study. We expect that the AOFAS Ankle-Hindfoot Score-DLV will prove valid and reliable, giving objective quantitative scores for patients' function and pain after trauma to the ankle or hindfoot. If the data confirm this, the instrument will be available for comparing outcome in future studies, and for comparing treatment outcome across hospitals or between patient groups. The SDC and Minimal Important Change (MIC) will especially reveal important information for sample size calculations in future studies.

    Three hospitals in the Netherlands will participate. Inclusion of patients has started in May 2014 and the expectation is to include all patients within 2 years for ankle injuries and 3 years for hindfoot injuries. With a maximum follow-up of 6.5 months, the presentation of data will be expected by end of 2016 and end of 2017, respectively.

    References

    1. ↵
      1. De Groot IB,
      2. Favejee MM,
      3. Reijman M, et al
      . The Dutch version of the Knee Injury and Osteoarthritis Outcome Score: a validation study. Health Qual Life Outcomes 2008;6:16. doi:10.1186/1477-7525-6-16
      OpenUrlCrossRefPubMed
    2. ↵
      1. Kuyvenhoven MM,
      2. Gorter KJ,
      3. Zuithoff P, et al
      . The foot function index with verbal rating scales (FFI-5pt): a clinimetric evaluation and comparison with the original FFI. J Rheumatol 2002;29:1023–8.
      OpenUrlAbstract/FREE Full Text
    3. ↵
      1. De Groot IB,
      2. Reijman M,
      3. Terwee CB, et al
      . Validation of the Dutch version of the Hip disability and Osteoarthritis Outcome Score. Osteoarthr Cartil 2007;15:104–9. doi:10.1016/j.joca.2006.06.014
      OpenUrlCrossRefPubMedWeb of Science
    4. ↵
      1. Shearer D,
      2. Morshed S
      . Common generic measures of health related quality of life in injured patients. Injury 2011;42:241–7. doi:10.1016/j.injury.2010.11.044
      OpenUrlCrossRefPubMed
    5. ↵
      1. Reininga IH,
      2. el Moumni M,
      3. Bulstra SK, et al
      . Cross-cultural adaptation of the Dutch Short Musculoskeletal Function Assessment questionnaire (SMFA-NL): internal consistency, validity, repeatability and responsiveness. Injury 2012;43:726–33. doi:10.1016/j.injury.2011.07.013
      OpenUrlCrossRefPubMed
    6. ↵
      1. Van Son MA,
      2. Den Oudsten BL,
      3. Roukema JA, et al
      . Psychometric properties of the Dutch Short Musculoskeletal Function Assessment (SMFA) questionnaire in patients with a fracture of the upper or lower extremity. Qual Life Res 2014;23:917–26. doi:10.1007/s11136-013-0529-z
      OpenUrl
    7. ↵
      1. Kitaoka HB,
      2. Alexander IJ,
      3. Adelaar RS, et al
      . Clinical rating systems for the ankle-hindfoot, midfoot, hallux, and lesser toes. Foot Ankle Int 1994;15:349–53. doi:10.1177/107110079401500701
      OpenUrlCrossRefPubMedWeb of Science
    8. ↵
      1. Madeley NJ,
      2. Wing KJ,
      3. Topliss C, et al
      . Responsiveness and validity of the SF-36, Ankle Osteoarthritis Scale, AOFAS Ankle Hindfoot Score, and Foot Function Index in end stage ankle arthritis. Foot Ankle Int 2012;33:57–63. doi:10.3113/FAI.2012.0057
      OpenUrlCrossRefPubMedWeb of Science
    9. ↵
      1. SooHoo NF,
      2. Vyas R,
      3. Samimi D
      . Responsiveness of the foot function index, AOFAS clinical rating systems, and SF-36 after foot and ankle surgery. Foot Ankle Int 2006;27:930–4. doi:10.1177/107110070602701111
      OpenUrlCrossRefPubMed
    10. ↵
      1. SooHoo NF,
      2. Shuler M,
      3. Fleming LL
      . Evaluation of the validity of the AOFAS Clinical Rating Systems by correlation to the SF-36. Foot Ankle Int 2003;24:50–5. doi:10.1177/107110070302400108
      OpenUrlCrossRefPubMedWeb of Science
    11. ↵
      1. Ibrahim T,
      2. Beiri A,
      3. Azzabi M, et al
      . Reliability and validity of the subjective component of the American Orthopaedic Foot and Ankle Society clinical rating scales. J Foot Ankle Surg 2007;46:65–74. doi:10.1053/j.jfas.2006.12.002
      OpenUrlCrossRefPubMed
    12. ↵
      1. Westphal T,
      2. Piatek S,
      3. Halm JP, et al
      . Outcome of surgically treated intraarticular calcaneus fractures—SF-36 compared with AOFAS and MFS. Acta Orthop Scand 2004;75:750–5. doi:10.1080/00016470410004148
      OpenUrlCrossRefPubMedWeb of Science
    13. ↵
      1. Pinsker E,
      2. Daniels TR
      . AOFAS position statement regarding the future of the AOFAS Clinical Rating Systems. Foot Ankle Int 2011;32:841–2. doi:10.3113/FAI.2011.0841
      OpenUrlCrossRefPubMedWeb of Science
    14. ↵
      1. Guyton GP
      . Theoretical limitations of the AOFAS scoring systems: an analysis using Monte Carlo modeling. Foot Ankle Int 2001;22:779–87. doi:10.1177/107110070102201003
      OpenUrlCrossRefPubMed
    15. ↵
      1. Beaton DE,
      2. Bombardier C,
      3. Guillemin F, et al
      . Guidelines for the process of cross-cultural adaptation of self-report measures. Spine 2000;25:3186–91. doi:10.1097/00007632-200012150-00014
      OpenUrlCrossRefPubMedWeb of Science
    16. ↵
      1. Guillemin F,
      2. Bombardier C,
      3. Beaton D
      . Cross-cultural adaptation of health-related quality of life measures: literature review and proposed guidelines. J Clin Epidemiol 1993;46:1417–32. doi:10.1016/0895-4356(93)90142-N
      OpenUrlCrossRefPubMedWeb of Science
    17. ↵
      1. Aaronson NK,
      2. Muller M,
      3. Cohen PD, et al
      . Translation, validation, and norming of the Dutch language version of the SF-36 Health Survey in community and chronic disease populations. J Clin Epidemiol 1998;51:1055–68. doi:10.1016/S0895-4356(98)00097-3
      OpenUrlCrossRefPubMedWeb of Science
    18. ↵
      1. Huh JW,
      2. Eun IS,
      3. Ko YC, et al
      . Reliability and validity of the Korean version of the Foot Function Index. J Foot Ankle Surg 2016;55:759–61. doi:10.1053/j.jfas.2016.03.011
      OpenUrl
    19. ↵
      1. Wu SH,
      2. Liang HW,
      3. Hou WH
      . Reliability and validity of the Taiwan Chinese version of the Foot Function Index. J Formos Med Assoc 2008;107:111–18. doi:10.1016/S0929-6646(08)60124-2
      OpenUrlCrossRefPubMedWeb of Science
    20. ↵
      1. Ware JE Jr.,
      2. Sherbourne CD
      . The MOS 36-item short-form health survey (SF-36). I. Conceptual framework and item selection. Med Care 1992;30:473–83.
      OpenUrlCrossRefPubMedWeb of Science
    21. ↵
      1. Mahabier KC,
      2. Van Lieshout EMM,
      3. Bolhuis HW, et al
      . HUMeral shaft fractures: measuring recovery after operative versus non-operative treatment (HUMMER): a multicenter comparative observational study. BMC Musculoskelet Disord 2014;15:39. doi:10.1186/1471-2474-15-39
      OpenUrlPubMed
    22. ↵
      1. Den Hartog D,
      2. Van Lieshout EMM,
      3. Tuinebreijer WE, et al
      . Primary hemiarthroplasty versus conservative treatment for comminuted fractures of the proximal humerus in the elderly (ProCon): a multicenter randomized controlled trial. BMC Musculoskelet Disord 2010;11:97. doi:10.1186/1471-2474-11-97
      OpenUrlCrossRefPubMed
    23. ↵
      1. De Haan J,
      2. Den Hartog D,
      3. Tuinebreijer WE, et al
      . Functional treatment versus plaster for simple elbow dislocations (FuncSiE): a randomized trial. BMC Musculoskelet Disord 2010;11:263. doi:10.1186/1471-2474-11-263
      OpenUrlCrossRefPubMed
    24. ↵
      1. Schep NWL,
      2. De Haan J,
      3. Iordens GIT, et al
      . A hinged external fixator for complex elbow dislocations: a multicenter prospective cohort study. BMC Musculoskelet Disord 2011;12:130. doi:10.1186/1471-2474-12-130
      OpenUrlPubMed
    25. ↵
      1. Pan SL,
      2. Liang HW,
      3. Hou WH, et al
      . Responsiveness of SF-36 and Lower Extremity Functional Scale for assessing outcomes in traumatic injuries of lower extremities. Injury 2014;45:1759–63. doi:10.1016/j.injury.2014.05.022
      OpenUrlCrossRefPubMed
    26. ↵
      1. Obremskey WT,
      2. Dirschl DR,
      3. Crowther JD, et al
      . Change over time of SF-36 functional outcomes for operatively treated unstable ankle fractures. J Orthop Trauma 2002;16:30–3. doi:10.1097/00005131-200201000-00007
      OpenUrlCrossRefPubMed
    27. ↵
      1. van Tetering EA,
      2. Buckley RE
      . Functional outcome (SF-36) of patients with displaced calcaneal fractures compared to SF-36 normative data. Foot Ankle Int 2004;25:733–8. doi:10.1177/107110070402501007
      OpenUrlCrossRefPubMed
    28. ↵
      1. Schep NWL,
      2. van Lieshout EMM,
      3. Patka P, et al
      . Long-term functional and quality of live assessment following post-traumatic distraction osteogenesis of the lower limb. Strategies Trauma Limb Reconstr 2009;4:107–12. doi:10.1007/s11751-009-0070-3
      OpenUrlPubMed
    29. ↵
      1. Garratt A,
      2. Schmidt L,
      3. Mackintosh A, et al
      . Quality of life measurement: bibliographic study of patient assessed health outcome measures. BMJ 2002;324:1417. doi:10.1136/bmj.324.7351.1417
      OpenUrlAbstract/FREE Full Text
    30. ↵
      1. Terwee CB,
      2. Bot SD,
      3. de Boer MR, et al
      . Quality criteria were proposed for measurement properties of health status questionnaires. J Clin Epidemiol 2007;60:34–42. doi:10.1016/j.jclinepi.2006.03.012
      OpenUrlCrossRefPubMedWeb of Science
    31. ↵
      1. Altman DG
      . Practical statistics for medical research. London: Chapman and Hall, 1991.
    32. ↵
      1. De Vet HCW,
      2. Terwee CB,
      3. Mokkink LB, et al
      . Measurement in medicine. New York: Cambridge University Press, 2011.
    33. ↵
      1. Hinkle DE,
      2. Wiersma W,
      3. Jurs SG
      . Applied statistics for the behavioral sciences. Boston, USA: Houghton MiZin, 1998.
    34. ↵
      1. Mokkink LB,
      2. Terwee CB,
      3. Patrick DL, et al
      . The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010;63:737–45. doi:10.1016/j.jclinepi.2010.02.006
      OpenUrlCrossRefPubMedWeb of Science
    35. ↵
      1. Streiner D,
      2. Norman G
      . Health measurement scales: a practical guide to their development and use. 2nd edn. Oxford, UK: Oxford Medical, 1996.
    36. ↵
      1. Nunnally J,
      2. Bernstein I
      . Psychometric theory. 3rd edn. New York, NY: McGraw-Hill, 1994.
    37. ↵
      1. De Vet HCW
      . Observer reliability and agreement. In: Armitage P, Colton T, eds. Encyclopedia of biostatistics. Boston: John Wiley & Sons Ltd, 1998:3123e8.
    38. ↵
      1. Stratford P
      . Reliability: consistency or differentiating among subjects? Phys Ther 1989;69:299–300.
      OpenUrlAbstract/FREE Full Text
    39. ↵
      1. Streiner DL,
      2. Norma GR
      . Health measurement scales. A practical guide to their development and use. New York: Oxford University Press, 2003.
    40. ↵
      1. McGraw KO,
      2. Wong SP
      . Forming inferences about some intraclass correlation coefficients. Psychol Methods 1996;1:30–46. doi:10.1037/1082-989X.1.1.30
      OpenUrlCrossRefWeb of Science
    41. ↵
      1. Scholtes VA,
      2. Terwee CB,
      3. Poolman RW
      . What makes a measurement instrument valid and reliable? Injury 2011;42:236–40. doi:10.1016/j.injury.2010.11.042
      OpenUrlCrossRefPubMed
    42. ↵
      1. de Vet HC,
      2. Terwee CB,
      3. Knol DL, et al
      . When to use agreement versus reliability measures. J Clin Epidemiol 2006;59:1033–9. doi:10.1016/j.jclinepi.2005.10.015
      OpenUrlCrossRefPubMedWeb of Science
    43. ↵
      1. Beckerman H,
      2. Roebroeck ME,
      3. Lankhorst GJ, et al
      . Smallest real difference, a link between reproducibility and responsiveness. Qual Life Res 2001;10:571–8. doi:10.1023/A:1013138911638
      OpenUrlCrossRefPubMedWeb of Science
    44. ↵
      1. de Vet HC,
      2. Bouter LM,
      3. Bezemer PD, et al
      . Reproducibility and responsiveness of evaluative outcome measures. Theoretical considerations illustrated by an empirical example. Int J Technol Assess Health Care 2001;17:479–87.
      OpenUrlPubMedWeb of Science
    45. ↵
      1. De Boer MR,
      2. De Vet HC,
      3. Terwee CB, et al
      . Changes to the subscales of two vision-related quality of life questionnaires are proposed. J Clin Epidemiol 2005;58:1260–8. doi:10.1016/j.jclinepi.2005.04.007
      OpenUrlCrossRefPubMedWeb of Science
    46. ↵
      1. Bland JM,
      2. Altman DG
      . Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1986;1:307–10. doi:10.1016/S0140-6736(86)90837-8
      OpenUrlCrossRefPubMedWeb of Science
    47. ↵
      1. McHorney CA,
      2. Tarlov AR
      . Individual-patient monitoring in clinical practice: are available health status surveys adequate? Qual Life Res 1995;4:293–307. doi:10.1007/BF01593882
      OpenUrlCrossRefPubMedWeb of Science
    48. ↵
      1. Guyatt GH,
      2. Deyo RA,
      3. Charlson M, et al
      . Responsiveness and validity in health status measurement: a clarification. J Clin Epidemiol 1989;42:403–8. doi:10.1016/0895-4356(89)90128-5
      OpenUrlCrossRefPubMedWeb of Science
    49. ↵
      1. Angst F,
      2. Verra ML,
      3. Lehmann S, et al
      . Responsiveness of five condition-specific and generic outcome assessment instruments for chronic pain. BMC Med Res Methodol 2008;8:26. doi:10.1186/1471-2288-8-26
      OpenUrlCrossRefPubMed
    50. ↵
      1. Cohen J
      . Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates, 1988.

    Footnotes

    • EMMVL and ASDB contributed equally.

    • Contributors EMMVL, ASDB, DEM, CHVdV, PTDH, WET and MHJV developed the study. ASDB and EMMVL drafted the manuscript. EMMVL will act as the principal trial investigator. ASDB, CHVdV, PTDH, DEM and MHJV will participate in patient inclusion and outcome assessment. ASDB, WET and EMMVL will perform statistical analysis of the study data. All authors have read and approved the final manuscript.

    • Competing interests None declared.

    • Patient consent Obtained.

    • Provenance and peer review Not commissioned; externally peer reviewed.