You are here

Article Text

Article menu

Download PDFPDF

XML
Rehabilitation medicine
Original research
Systematic review of the measurement properties of performance-based functional tests in patients with neck disorders
  1. Steven McGee1,
  2. Taylor Sipos1,
  3. Thomas Allin1,
  4. Celia Chen1,
  5. Alexandra Greco1,
  6. Pavlos Bobos1,2,3,
  7. Joy MacDermid1,2,4
  8. CATWAD
    1. 1 School of Physical Therapy, Health and Rehabilitation Sciences, Western University, London, Ontario, Canada
    2. 2 Western's Bone and Joint Intitute, Western University, London, Ontario, Canada
    3. 3 Dalla Lana School of Public Health, Institute of Health Policy Management and Evaluation, Department of Clinical Epidemiology and Health Care Research, University of Toronto, Toronto, Ontario, Canada
    4. 4 School of Rehabilitation Science, McMaster University, Hamilton, Ontario, Canada
    1. Correspondence to Dr Pavlos Bobos; pbobos{at}uwo.ca

    Abstract

    Objectives The purpose of this systematic review is to identify and synthesise studies evaluating performance-based functional outcome measures designed to evaluate the functional abilities of patients with neck pain.

    Design Systematic review.

    Data sources A literature search using PubMed, Scopus, CINAHL, EMBASE, COCHRANE, Google Scholar and a citation mapping strategy was conducted until July 2019.

    Eligibility criteria More than half of the study’s patient population had neck pain or a musculoskeletal neck disorder and completed a functional-based test. Clinimetric properties of at least one performance-based functional tests were reported. Both traumatic and non-traumatic origins of neck pain were considered.

    Data extraction and synthesis Relevant data were then extracted from selected articles using an extraction guide. Selected articles were appraised using the Quality Appraisal for Clinical Measurement Research Reports Evaluation Form (QACMRR).

    Results The search obtained 12 articles which reported on four outcome measures (functional capacity evaluations (FCE), Baltimore Therapeutic Equipment Work Simulator II (BTEWS II), Functional Impairment Test-Hand and Neck/Shoulder/Arm (FIT-HaNSA)) and a physiotherapy test package, to assess the functional abilities in patients with mechanical neck pain. Of the selected papers: one reports content validity, five construct validity, four reliability, one sensitivity to change and one both reliability and construct validity. QACMRR scores ranged from 68% to 95%.

    Conclusions This review found very good quality evidence that the FIT-HaNSA has excellent inter and intra-rater reliability and very weak to weak convergent validity. Excellent quality evidence of fair test-retest reliability, weak convergent validity and very weak known groups validity for the BTEWS II test was found. Good to excellent quality evidence exists that an FCE battery has poor to excellent reliability and very weak to strong validity. Good to excellent quality of weak to strong validity and trivial to strong effect sizes were found for a physiotherapy test package.

    Prospero registration number CRD42018112358

    • functional
    • psychometric properties
    • neck
    • cervical
    • outcome measures

    This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

    https://doi.org/10.1136/bmjopen-2019-031242

    Statistics from Altmetric.com

      Request Permissions

      If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

      Strengths and limitations of this study

      • The psychometric properties of performance outcome measures for neck pain were synthesised and critically appraised.

      • This study assessed the risk of bias and the quality of measurements properties.

      • The feasibility or usability of these tools was not assessed.

      Introduction

      Neck pain has been associated with high disability and is regarded as a substantial societal burden.1 Approximately 70% of people experience neck pain within their lifetime and about 33% of adults experience neck pain every year.2 3 Further concern is warranted as it has been suggested that the incidence of neck pain is increasing.4–6 The economic burden due to neck disorders is high, including lost wages, costs of treatment and compensation expenditures to injured people.7 8 Neck pain is second only to low back pain in annual workers’ compensation costs in the USA and has been associated with many other comorbidities such as headaches, anxiety, depression, back pain and arthralgias.6 9 10

      Outcome measures are a crucial component in monitoring patients with neck pain to determine the effects of treatment,11 12 evaluation of interventions, guiding return to work and justifying treatment.13 14 Several self-reported outcome measures currently exist to assess disability and function in those with neck pain (eg, the Neck Disability Index-NDI).13 Evidence-based clinical practice guidelines suggest that measures assessing physical performance should also be used for people with neck pain.15 Performance-based testing is where the assessment is based on actual performance of a task or activity. Physical performance can be assessed by testing a person’s ability to execute a standardised activity in a standardised environment (ie, clinical setting).16 Time to complete the activity, number of repetitions performed and weight lifted are frequently used to quantify the physical performance.17 Conversely, self-report measures examine patients’ perception and experience of their ability to perform functional tasks.16 Previous research has demonstrated poor to fair relationships between physical performance and self-report measures of ability in patients with various musculoskeletal disorders suggesting that these measures assess different constructs of function.17 18 Consequently, physical performance tests and self-report measures complement each other and may each contribute unique information about a patient’s function.19

      A fundamental component of monitoring outcomes is having reliable and valid tools with known measurement properties.13 20 While recent research has investigated the psychometric properties of patient-reported outcomes in people with neck pain,13 21 there is a gap in knowledge with respect to performance-based functional outcomes. The purpose of this systematic review was to identify and synthesise clinical measurement studies that evaluate measurement properties of performance-based functional tests in patients with neck disorders.

      METHODS

      Patient and public involvement

      There was no patient or public involvement in the design or planning of this study.

      Study design and protocol registration

      We conducted a systematic review to evaluate the psychometric properties of performance-based functional tests for people with mechanical neck disorders. The protocol was registered in PROSPERO register with registration number CRD42018112358.

      Search strategy

      A database search using CINAHL, PubMed, Scopus and Google Scholar was performed to identify articles published until July 2019. The following search strategy was used to search all databases for eligible studies: (Reliability OR validity OR responsiveness OR calibration OR validation) OR (minimal detectable change) OR (clinically important difference) OR (psychometric properties) AND cervical OR neck OR c-spine AND (performance measure) OR (functional test) OR (functional outcome) OR (performance outcome). MeSH terms were searched in PubMed. A citation map of articles and systematic reviews selected for the full-text review was performed. This strategy was included to minimise the risk of publication bias. The full search strategy is summarised in online supplementary appendix 1. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) process22 was followed to ensure all appropriate steps were taken in the selection process (figure 1).

      Supplemental material

      Figure 1
      Figure 1

      Selection of the studies for inclusion in the systematic review.

      Inclusion criteria

      Articles were included in the final review if all of the following criteria were met:

      • >50% of the study’s patient population had neck pain or a musculoskeletal neck disorder (eg, whiplash associated disorder (WAD II))

      • Patients in the study completed a functional-based test

      • Clinometric properties of at least one performance-based test were reported.

      A test was considered functional-based if it met the following criteria:

      • Assessment of a patient’s ability to execute a standardised activity in a standardised environment

      • Tests assessing muscular endurance (eg, cervical flexion test) or proprioception were not deemed functional-based as they are often not reflective of physical working conditions.

      Both traumatic and non-traumatic origins of neck pain were considered. Definitions for the properties can be found in online supplementary appendix A.

      Supplemental material

      Article selection

      Titles and abstracts generated by the search strategy were screened by two authors (SM and PB) independently. Articles that met the inclusion criteria and selected for a full-text review were also reviewed in pairs of authors. Disagreements were resolved by the most experienced author (JCM)

      Data extraction

      Data extraction and critical appraisal were performed in pairs of two raters among the authors, after the completion of a calibration session in which the most experienced author (JM) reviewed the data extraction tools with the authors that performed the data extraction. When reviewers disagreed during data extraction and/or critical appraisal, and consensus could not be met, a third author arbitrated. A data extraction form23 (see online supplementary appendix A and B), developed by one of the authors (JM), was used to ensure systematicity. Authors extracted sample size, patient population characteristics, functional tests performed and reported psychometric properties. The interpretation of ICC was as follows: ICC<0.50 indicating poor, 0.50≤ICC<0.75 indicating moderate, 0.75<ICC<0.9 indicating good and ICC>0.9 indicating excellent reliability were used as a common benchmark.24 For validity estimates, correlation coefficient (Pearson’s/Spearman) and the 95% CI were extracted if were available.23 25 Evan’s guidelines to interpret the strength of the correlation was used which included: 0.00–0.19 ‘very weak’, 0.20–0.39 ‘weak’, 0.40–0.59 ‘moderate’, 0.60–0.79 ‘strong’ and 0.80–1.00 ‘very strong’.26 To assist clinical decision making, standard benchmark scores of trivial (<0.20), small (≥0.20 to<0.50), moderate (≥0.50 to<0.80) or large (≥0.80), as proposed by Cohen, were used.27 For studies assessing construct validity specifically, results in accordance with predefined hypotheses were evaluated to interpret the findings.

      Quality appraisal for clinical measurement research reports evaluation form

      Pairs of authors critically appraised the quality of each study using a standardised 12-item evaluation tool (QACMRR) designed to assess the quality of studies determining measurement properties in outcome measures (see online supplementary appendix C). If disagreement was present, a third person (JM) assisted in resolving the discrepancy.23 This tool has been found to have moderate to excellent preconsensus inter-rater reliability (ICC: 0.69–0.91, κ=0.62–1.00) across a number of systematic reviews.23 25 28 The evaluation criteria of this tool included 12 items: (1) thorough literature review to define the research question; (2) specific inclusion/exclusion criteria; (3) specific hypotheses; (4) appropriate scope of psychometric properties; (5) sample size; (6) follow-up; (7) the authors referenced specific procedures for administration, scoring and interpretation of procedures; (8) measurement techniques were standardised; (9) data were presented for each hypothesis; (10) appropriate statistics-point estimates; (11) appropriate statistical error estimates; and (12) valid conclusions and recommendations.23 25 Each item is scored from 0 to 2 with (score=2) is the best; (score=1) is acceptable but suboptimal; (score=0) is not done/documented, substantially inadequate or inappropriate. An article’s total score, quality, was calculated by the sum of scores for each item, divided by the numbers of items and multiplied by 100%.23 25 Overall, the quality summary of appraised articles ranges from (0%–30%) poor, (31%–50%) fair, (51%–70%) good, (71%–90%) very good and (>90%) excellent.

      Results

      The search strategy resulted in 840 published articles. After duplications were removed, 31 articles were deemed relevant and were screened at full text. Overall, 12 articles met our inclusion criteria (figure 1). The excluded articles were removed due to inappropriate patient populations, investigations into self-report measures or tests assessing proprioception/muscular endurance rather than functional-based measures, or because the articles were found to be systematic reviews. The characteristics of the included studies and the summary of psychometric properties are presented in table 1. The quality assessment is summarised and presented in table 2. Percent agreement was calculated for quality scores between the two raters and it was 90%.

      View this table:
      Table 1

      Summary of studies reporting psychometric properties of functional-based tests in neck disorder patients

      View this table:
      Table 2

      Quality of studies on psychometric properties of functional-based tests evaluated in neck disorder patients

      Participants

      Participants in the selected articles had various types of neck pain including subacute, chronic and whiplash-associated disorder. The mean/median age of the samples of each study ranged from 30 to 48 years of age. The proportion of women in each article ranged from 34% to 78% of the study population. Two studies that had a mixed sample of subjects with various spinal pain did not report the demographics of the neck pain portion of their sample. One study did not contain any subjects and performed a review of epidemiological literature to establish content validity for work-related neck disorders (table 1).

      Functional-based tests

      The 12 articles that were included for review provided properties on the following functional based tests: functional capacity evaluations (FCE),29–34 The Baltimore Therapeutic Equipment work simulator II (BTEWS II),35 Functional Impairment Test- Hand and Neck/Shoulder/Arm (FIT-HaNSA),36 as well as items off of a physiotherapy test package including a cervical and lumbar Progressive Isoinertial Lifting Evaluation (PILE-C, PILE-L) test37–40 and 2×20 m with burden walking test (2×20M-WWB).37–40 Descriptions of all functional-based tests and their relevant subtasks are provided in online supplementary appendix D.

      Functional capacity evaluations

      Six articles reported measurement properties for an FCE battery. We identified multiple versions of the FCE in the literature with one article reporting properties on the Workwell FCE,30 two reporting on the Whiplash Associated Disorder (WAD) FCE29 31 and three reporting on the neck-FCE.32–34 These test batteries include various combinations of muscular strength, endurance and functional based tests. The measurement properties of the functional based tests used by the FCE are outlined in table 3.

      View this table:
      Table 3

      Psychometric properties of the functional capacity evaluation

      Individuals with subacute to chronic WAD

      Trippolini et al (2014)30 evaluated the Workwell FCE test-retest reliability, measurement error, convergent validity and predictive criterion validity of future work capacity in workers diagnosed with WAD I or II. Interclass correlation coefficients (ICC) ranged from 0.66 to 0.96 (moderate to excellent). Limits of agreement relative to mean performance ranged from 21% to 57% for functional based sub-tests. Correlations between FCE sub scores and baseline work capacity were very weak to weak ranging between r=0.06 and r=0.39. FCE sub scores did not predict future work capacity at 1, 3, 6 and 12 months.

      Trippolini et al (2015)29 assessed the WAD FCE (31) and evaluated convergent validity and known-groups validity. FCE subscales showed very weak to strong correlations (0.15–0.68) with each of: pain, self-reported functional ability, self-reported disability, anxiety and depression. It was found that the FCE had known-group sex validity (males vs females) for 1 of 3 functional subtests (lifting waist-overhead) and reported significant performance differences between culture groups (German vs non-German language groups). To test construct validity, 29 a priori formulated hypotheses were tested, 4 related to gender differences, 20 related associations with other constructs, 5 related to cultural differences. In total 23 out of 29 hypotheses were confirmed (79 %).

      Work-related neck disorders

      Reesink et al. (2007)34 developed an independent FCE for patients with musculoskeletal neck disorders (neck FCE). They performed a review of epidemiological literature and identified four physical risk factors for work-related neck disorders and used that information to develop an FCE consisting of eight functional-based tests. Content validity was established by following operational definitions of the risk factors when searching the literature and using current literature to provide a rationale to guide their development of the tasks comprising the FCE.

      Chronic neck pain

      Reneman et al. (2017)32 measured test-retest reliability of the subscales of the neck FCE in patients with multifactorial neck pain. Test-retest ICC’s ranged from poor to excellent (0.39–0.96). Limits of agreement relative to mean performance range from 32.0% to 56.5% for functional based sub tests. Convergent validity was performed against the Neck Disability Index (NDI) items and total score.33 The authors found weak to strong Pearson correlations (0.39–0.70) for the FCE sub scores to both NDI individual items and the NDI total score.

      The BTEWS II

      Chronic neck pain

      Lomond and Côté, (2011)35 reported on the reliability, measurement error, minimum detectable change (MDC) and validity of the power output (PO) task during the BTEWS II test in patients with chronic neck and shoulder pain (table 4). Test-retest reliability, measured with Spearman Rank correlations and ICC’s was moderate and measured at ⍴=0.37 and ICC2,1 = 0.54, respectively. The SE of measurement (SEM) and the minimal detectable change at 90% confidence (MDC90) for the PO task were measured as 30.25 and 70.59, respectively. Weak Spearman Rank correlations between the PO task and the NDI, Shoulder Pain and Disability Index (SPADI) and Numeric Rating Scale (NRS) for pain tests were recorded. There were no significant performance differences between control and pain groups for the PO task.

      View this table:
      Table 4

      Summary of Fit-HaNSA’s psychometric properties in neck disorder patients

      Functional Impairment Test-Hand and Neck/Shoulder/Arm

      Subacute to chronic WAD

      Pierrynowski et al (2016)36 reported on the reliability, measurement error, MDC and validity of the Functional Impairment Test-Hand and Neck/Shoulder/Arm (Fit-HaNSA) test in a sample of people with WAD II following motor vehicle collision (MVC) (table 5). Intra-rater reliability ICC’s for patient subtask and total scores were moderate to good ranging between 0.70–0.78.36 Inter-rater reliability ICC’s for patient subtask and total scores were moderate to good and ranged between 0.54 and 0.84.36 The Bland and Altman plot for the patient group showed a 26 s (s) bias in terms of improved performance on the second test (possible learning effect). The SD of difference was 124 s and 95% Limits of Agreement (LoA95) was 248 s.36 The SEM for people with WAD II was reported to be 76 s. The MDC90 was measured as 176 s.36

      View this table:
      Table 5

      Psychometric properties of Baltimore Therapeutic Equipment work simulator II–Power Output task

      Spearman rank correlations were also calculated between the Fit-HANSA, Numeric Pain Rating Scale (NPRS), NDI, the disabilities of arm, hand and shoulder (DASH) and six cervical range of motion measures. Most (59 of 78) of the correlations between performance and comparator measures were very weak to weak (r=<0.4).36 All correlations between total Fit-HaNSA scores and subtask scores had good correlations (r=<0.75), except for Task 1-Task 3.36 Significant performance differences between WAD II and control groups (known group validity) were recorded for the total Fit-HaNSA score and all three subtask scores.36

      Physiotherapy test package subtests

      Ljungquist et al published a series of articles37–40 which evaluated the clinimetric properties of a physiotherapy test package for patients with spinal pain (table 6). This package included muscular strength & endurance tests, submaximal endurance tests, and three functional tests. These functional tests included the PILE-C, PILE-L, and 2×20M-WWB test. Ljungquist’s series of articles reported on convergent validity, known-groups validity, reliability, measurement error and sensitivity to change for these tests.37–40

      View this table:
      Table 6

      Psychometric properties of performance-based tests included in physiotherapy test package

      Undetermined duration of neck pain

      In a 1999 article,39 correlations between the tests of the package and pain (CR-10) and perceived exertion (Borg RPE) were determined. All correlations were very weak to moderate (0.10–0.48) except for moderate to strong correlations (0.55–0.65) between the PILE-C test and pain intensity and between 2×20M-WWB test and pain intensity.

      In a 2003 article,37 the PILE-C, PILE-L and 2×20M-WWB tests were tested to determine their ability to discriminate between known-groups (neck pain vs back pain). Subjects with spinal pain completed the CR-10, the University of Alabama Pain Behaviour Scale (UAB) and the Borg RPE test. Specific cut points were used to distinguish patients with high vs low pain intensity, high vs low pain behaviour, and high versus low perceived exertion in patients, respectively. Participants then completed the test package and it was determined if each subtest could discriminate between participants with high vs low pain intensity. The PILE-C and the 2×20M-WWB tests were hypothesised to be more difficult for persons with neck pain and the PILE-L was hypothesised to be more difficult for persons with back pain. Subjects with neck pain performed worse on the PILE-C test compared with those with back pain. Subjects with back pain did not perform worse than those with neck pain on the PILE-L test and subjects with back pain performed worse on the 2×20M-WWB test.

      The functional tests were able to discriminate between all three subgroups with the exception of the PILE-C being unable to discriminate between participants with high versus low perceived exertion.

      In a paper from 1999,39 the PILE-C, PILE-L and 2×20M-WWB tests were found to have significant discriminative abilities in distinguishing healthy subjects from patients with spinal pain. The sensitivity and specificity for this known group discrimination for the PILE-C test, were reported to be 0.93 (very strong) and 0.69 (strong), respectively. The sensitivity and specificity for the PILE-L test were reported to be 0.85 (very strong) and 0.65 (strong), respectively.

      The inter and intra rater reliability were tested on participants with spinal pain.38 Limits of agreement were used to measure inter rater reliability and repeatability, defined as 2x the within-subject SD of each variable. Interrater agreement for two tests was deemed ‘acceptable’, while all three functional tests had ‘clinically acceptable’ intra-rater reliability.

      Sensitivity-to-change was evaluated in the test package following 6 months of a physiotherapy intervention. Using ROC curves, Wilcoxon sign ranked tests and spearman correlation coefficients, only the 2×20m-WWB test and the PILE-C (women only) were deemed to be sensitive to change.40 Additionally, moderate to large effect sizes were found for all test components.

      Discussion

      This study synthesised 12 studies assessing clinometric properties of 4 different functional-based assessments. Given the limited number of studies, the substantial variation in the types of tests examined, the methods used to assess the clinical measurement properties, and the study populations, the current state of knowledge does not allow firm conclusions regarding recommendations for an optimal functional-based test at this time. Overall, the quality ranging from good to excellent (67%–92%,) as determined by the QACMRR, for a range of properties of the four different assessments in patients with acute or chronic neck pain that is musculoskeletal in origin. Studies obtaining higher percentages indicate research that has been consistent with best practice where studies with lower percentages are more likely to be inadequate or inappropriate

      Functional capacity evaluation

      The breadth of a functional-based test is variable and defined by the developers. An advantage of the functional assessment designed by Reesink et al 34 is that they mapped the eight subtests to risk factors identified in the literature for work-related neck disorders. The eight subtests consist of: material handling tasks, lifting floor to waist, overhead lift test, one-handed and two-handed carrying, overhead working, repetitive reaching, overhead lifting, and repetitive bending and overhead reaching. Given the systematic approach and rationale these authors used in developing the FCE and this approach being used in previous research,41 we suggest that this test has strong content validity.

      Six articles address the clinical measurement properties of this FCE ranging from good to excellent quality (67%–92%). There was evidence that the FCE was stable over test-retest time of 7–14 days.31 32 These measures demonstrate longer stability over time compared with self-report measures such as the Neck Disability Index (NDI) which has demonstrated test-retest reliability within only a short period of 0–3 days.28 Whether this longer-term stability is a characteristic of functional-based tests or reflects differences in study populations in context requires further testing. These two studies had relatively lower quality scores on the QACMRR (67%–75%) compared with other studies in this review putting into question test-retest time. Although test-retest reliability has been assessed, inter-rater and intra-rater reliability has yet to be researched. Unlike self-report measures, we expect measurement error due to the evaluator and functional-based tests. Thus, future research should explore these aspects of reliability.

      Convergent validity is often examined in clinical measurement studies. We suggest that this may be because these comparisons are easily performed by correlating different tests rather than providing strong confidence in the validity of the measurement. Often convenient comparisons are performed rather than those most relevant. Across many domains and measures it has become clear that the relationship between self-reported function and performance-based function or physical impairment is often very weak to moderate. Therefore, the value of assessment of these relationships as a form of validation has limited value. Several studies of very good to excellent quality have reported on the convergent validity of the FCE.29 30 33 The highest quality article determined by the QACMRR (92%) found the relationship between the FCE and work capacity to be poorly associated with one another.30 The same study found that the ability of the FCE to predict future work capacity was poor. This may be considered a more important comparison since ideally functional-based tests would relate to important outcomes like return to work. No studies to our knowledge report the responsiveness or sensitivity to change of the FCE. This is an important gap since the focus of rehabilitation is often to remediate limitations in goal impairments or work capacity, and assessment of these changes is critical to clinical decision-making and reporting outcomes. Thus, future research should evaluate the responsiveness of the FCE to provide insight in the measure’s ability to detect change after an intervention.

      Functional Impairment Test-Hand and Neck/Shoulder/Arm

      One study of very good quality (88%) assessed the FIT-HaNSA, a test consisting of two reaching tasks (waist and eye-level) and sustained overhead task performance.36 Overall, the FIT-HaNSA demonstrated excellent inter-rater reliability (0.84) and intra-rater reliability (0.78). The specific subtests included within the FIT-HaNSA similarly demonstrate fair to excellent (0.54–0.80) and good (0.70–0.72) inter-rater and intra-rater reliability respectively. The FIT-HaNSA also demonstrated a clear ability to distinguish between people with WAD two and healthy controls. Correlations between the FIT-HaNSA and other patient self-report disability and functional outcome measures (NPRS, NDI, DASH, CROM and FIT-HaNSA) were generally very weak to weak (ρ<0.4), consistent with other studies comparing performance and self-report.17 18 The largest limitation in critically synthesising information for this test is that only a single study was found that reported the measurement properties for people with neck disorders. It should be noted however that it has been validated in other MSK disorders.35 41 Although others have noted the lag in development of functional-based measures in comparison to self-report measures, FIT-HaNSA was recommended as a functional-based measure for people with shoulder disorders.42 Further research is necessary to investigate the responsiveness of the FIT-HaNSA.

      Baltimore Therapeutic Equipment work simulator II

      Another study of very good quality (88%) assessed the efficacy of the BTEWS II where the participants performed a dynamic pushing and pulling task in which power output was recorded over a 10 s sample.35 While the convergent validity aspect of this paper was assessed as consistent with best practice through the critical appraisal process, the relationship between the power output on the BTEWS and measures of pain and disability (NDI, SPADI, NRS) were poorly associated with each other. In addition, the power output component was not found to be significantly different between people with neck pain and healthy controls which suggests it might not be discriminative. Discrimination between patients and healthy controls is a low standard for an outcome measure, and tests that cannot fulfil this benchmark should be viewed with caution. Because of the weak measurement properties demonstrated by the power output component of the BTEWS II, it does not appear to be a desirable functional-based measure to assess function in people with neck pain. However, we acknowledge for all of the functional-based tests the evidence pool is so shallow that there is high potential that future studies might lead to different conclusions. Future research should also investigate the reliability and responsiveness of the BTEWS II.

      Physiotherapy test package subtests

      Four studies ranging from good to very good quality (68%–82%) assessed relevant items from a physiotherapy test package, including a lift from floor-to-waist and a waist-to-shoulder task and a two-handed carrying task. The properties of these assessment items include weak to moderate correlations to pain, perceived exertion, and had “fair to good” reliability. The 2×20m-WWB and PILE-C tests were found to be sensitive-to-change which is valuable information as no other study has assessed this property in functional-based measures in patients with neck disorders. Thus, this measure may be of value in clinical settings when assessing functional capacity before and after a treatment intervention. All tests had discriminative ability for detecting participants with spinal pain vs healthy controls. Most of the three tests demonstrated poor construct validity in that they were poorly related to pain and perceived exertion and the results were not in accordance with pre-defined hypotheses. Thus, further research is necessary to investigate these constructs. Three of the four results from the studies assessing the physiotherapy test package had a mixed sample of patients with various pain sites including back pain. While the majority of each cohort in these studies had neck pain, careful consideration should be taken to apply these tests to a neck pain specific population.

      Clinical implications

      This study confirms that functional-based tests have had far less development and evaluation than self-report measures. Limitations include the number of tests and insufficient body of evidence to make confident recommendations with respect to functional-based testing. It is clear that self-report and functional-based measures provide different perspectives. Theoretically, functional-based tests are important to inform our understanding about the mechanisms of intervention and how interventions increase capacity. Future research may benefit by also comparing results from a functional-based measure to work capacity to when assessing construct validity. Overall more work is required to further establish the psychometric properties of functional-based tests in persons with neck disorders, including sensitivity-to-change, responsiveness, and predictive validity.

      The FCE evaluated patients with neck pain of varying origin including WAD, work-related neck disorders, and chronic idiopathic neck pain. The BTEWs II evaluated functional capacity in patients with chronic neck pain, the FIT-HaNSA evaluated patients with WAD, and the physiotherapy test package did not specify the origin of musculoskeletal neck pain in their cohort. Thus, specific functional-based measures may be more applicable depending on the origin of the musculoskeletal neck pain being assessed.

      The data presented suggest that the FIT-HaNSA has the strongest clinometric properties though this is based on a single higher quality paper specific to neck disorder.36 Importantly, normative data have been published,43 it has been validated in multiple studies in patients with shoulder conditions44–46 and has been recommended when compared with other measures.42 The FCE has a limited evidence base from which to draw, though it was developed with strong content validity and further evaluation may demonstrate its usefulness.

      Limitations

      A challenge in synthesising clinical measurement evidence is the wide range of properties and indicators that need to be considered. Unlike effectiveness studies where one can focus on the effect size of treatment there are many considerations that would affect the recommendations made about outcome measures. This is further complicated when the pool of evidence is shallow. Although the quality assessment tool (QACMRR) developed by one of the authors of this review which assess the quality of design of individual studies were useful for interpreting the evidentiary pool, there is no clear method to synthesise the extracted clinical measurement evidence. While some systematic reviews on treatment might only report findings from high-quality studies, it is important to see how outcome measures perform in different contexts. Further, the assessment of quality is complicated given that clinical measurement studies have so many dimensions. Therefore, exclusion of lower quality studies has questionable value. Thus, a more practical approach is to consider quality when interpreting the findings, rather than excluding studies.

      The QACMRR focuses on whether the authors made appropriate decisions in selecting the scope and methods of their clinical measurement evaluations within a given study and provides descriptors of poor fair or good design options. Quality focuses on issues that might affect risk of bias or imprecision in estimates; whereas risk of bias assessments focusses on items that might result in a biassed estimate. For example, insufficient power is a precision (quality) issue, not a risk of bias. Although it is difficult to interpret the meaning of the percentage of the QACMRR as there are no established cut-offs for distinguishing good and poor-quality studies, it provides one way of ranking the articles in order of quality. We did not use COSMIN checklist since it was developed for PROMS and some of the components/steps that involved are not applicable to performance-based tests.

      Another limitation in this review was that the feasibility or usability of these tools was not assessed. While feasibility was not the focus of this review, information on the practical application of these functional-based measures provides valuable information to clinicians for determining whether these tests are appropriate to use in their given setting. Thus, future research should not only investigate further the psychometric properties of these tools, but also report the feasibility of using these tests so that they may be used in clinical settings and to identify limitations that restrict their application in practice.

      Conclusion

      This review found very good quality evidence that the FIT-HaNSA has excellent inter and intra-rater reliability and very weak to weak convergent validity. Excellent quality evidence of fair test-retest reliability, weak convergent validity, and very weak known groups validity for the BTEWS II test was found. Good to excellent quality evidence exists that an FCE battery has poor to excellent reliability and very weak to strong validity. Good to excellent quality of weak to strong validity and trivial to strong effect sizes were found for a physiotherapy test package. Functional-based evaluation in people with neck disorders is an area needing much research attention both to establish the measurement properties of existing measures, potentially to develop innovative new measures and to perform head-to-head comparisons of measures before an optimal functional-based test can be identified.

      References

      1. ↵
        2. Carroll LJ ,
        3. Hogg-Johnson S ,
        4. van der Velde G , et al
        . Course and prognostic factors for neck pain in the general population: results of the bone and joint decade 2000-2010 Task force on neck pain and its associated disorders. J Manipulative Physiol Ther 2009;32:S87-96.doi:10.1016/j.jmpt.2008.11.013
      2. ↵
        2. Croft PR ,
        3. Lewis M ,
        4. Papageorgiou AC , et al
        . Risk factors for neck pain: a longitudinal study in the general population. Pain 2001;93:317–25.doi:10.1016/S0304-3959(01)00334-7
        OpenUrlCrossRefPubMedWeb of Science
      3. ↵
        2. VosT ,
        3. AllenC ,
        4. AroraM , et al
        . Global, regional, and national incidence, prevalence, and years lived with disability for 310 diseases and injuries, 1990-2015: a systematic analysis for the global burden of disease study 2015. Lancet 2016;388:1545–602.doi:10.1016/S0140-6736(16)31678-6
        OpenUrlCrossRefPubMed
      4. ↵
        2. Blanpied PR ,
        3. Gross AR ,
        4. Elliott JM , et al
        . Neck pain: revision 2017. J Orthop Sports Phys Ther 2017;47:A1–83.doi:10.2519/jospt.2017.0302
        OpenUrlCrossRef
      5. ↵
        2. Nygren A ,
        3. Berglund A ,
        4. von Koch M
        . Neck-and-shoulder pain, an increasing problem. strategies for using insurance material to follow trends. Scand J Rehabil Med Suppl 1995;32:107–12.
        OpenUrlPubMed
      6. ↵
        2. Wright A ,
        3. Mayer TG ,
        4. Gatchel RJ
        . Outcomes of disabling cervical spine disorders in compensation injuries. A prospective comparison to tertiary rehabilitation response for chronic lumbar spinal disorders. Spine 1999;24:178-83.doi:10.1097/00007632-199901150-00020
      7. ↵
        2. Rempel DM ,
        3. Harrison RJ ,
        4. Barnhart S
        . Work-Related cumulative trauma disorders of the upper extremity. JAMA 1992;267.doi:10.1001/jama.1992.03480060084035
      8. ↵
        2. Borghouts JA ,
        3. Koes BW ,
        4. Vondeling H , et al
        . Cost-Of-Illness of neck pain in the Netherlands in 1996. Pain 1999;80:629–36.doi:10.1016/S0304-3959(98)00268-1
        OpenUrlCrossRefPubMedWeb of Science
      9. ↵
        2. Hogg-Johnson S ,
        3. van der Velde G ,
        4. Carroll LJ , et al
        . The burden and determinants of neck pain in the general population: results of the bone and joint decade 2000-2010 Task force on neck pain and its associated disorders. J Manipulative Physiol Ther : 2009;32:S46-60.doi:10.1016/j.jmpt.2008.11.010
      10. ↵
        2. Bobos P ,
        3. Nazari G ,
        4. Palimeris S , et al
        . The contribution of health and psychological factors in patients with chronic neck pain and disability: a cross-sectional study. JCDR : 2018.doi:10.7860/JCDR/2018/31284.11203
      11. ↵
        2. Bobos P ,
        3. Billis E ,
        4. Papanikolaou D-T , et al
        . Does deep cervical flexor muscle training affect pain pressure thresholds of myofascial trigger points in patients with chronic neck pain? A prospective randomized controlled trial. Rehabil Res Pract 2016;2016:1–8.doi:10.1155/2016/6480826
        OpenUrl
      12. ↵
        2. Nazari G ,
        3. Bobos P ,
        4. Billis E , et al
        . Cervical flexor muscle training reduces pain, anxiety, and depression levels in patients with chronic neck pain by a clinically important amount: a prospective cohort study. Physiother Res Int 2018;23.doi:10.1002/pri.1712
      13. ↵
        2. Bobos P ,
        3. MacDermid JC ,
        4. Walton DM , et al
        . Patient-Reported outcome measures used for neck disorders: an overview of systematic reviews. J Orthop Sports Phys Ther 2018;48:775–88.doi:10.2519/jospt.2018.8131
        OpenUrl
      14. ↵
        2. MacDermid JC ,
        3. Walton DM ,
        4. Bobos P , et al
        . A qualitative description of chronic neck pain has implications for outcome assessment and classification. Open Orthop J 2016;10:746–56.doi:10.2174/1874325001610010746
        OpenUrl
      15. ↵
        2. Childs JD ,
        3. Cleland JA ,
        4. Elliott JM , et al
        . Neck pain: clinical practice guidelines linked to the International classification of functioning, disability, and health from the orthopedic section of the American physical therapy association. J Orthop Sports Phys Ther 2008;38:A1-A34.doi:10.2519/jospt.2008.0303
      16. ↵
        2. Kay TM ,
        3. Huijbregts M
        . Physical rehabilitation outcome measures: a guide to enhanced clinical decision making. Second Edition. Canada: Physiother, 2003.
      17. ↵
        2. Simmonds MJ ,
        3. Olson SL ,
        4. Jones S , et al
        . Psychometric characteristics and clinical usefulness of physical performance tests in patients with low back pain. Spine 1998;23:2412–21.doi:10.1097/00007632-199811150-00011
        OpenUrlCrossRefPubMedWeb of Science
      18. ↵
        2. Stratford PW ,
        3. Kennedy D ,
        4. Pagura SMC , et al
        . The relationship between self-report and performance-related measures: Questioning the content validity of timed tests. Arthritis Rheum 2003;49:535–40.doi:10.1002/art.11196
        OpenUrlCrossRefPubMedWeb of Science
      19. ↵
        2. Novy DM ,
        3. Simmonds MJ ,
        4. Lee CE
        . Physical performance tasks: what are the underlying constructs? Arch Phys Med Rehabil 2002;83:44–7.doi:10.1053/apmr.2002.27397
        OpenUrlCrossRefPubMedWeb of Science
      20. ↵
        2. MacDermid JC ,
        3. Stratford P
        . Applying evidence on outcome measures to hand therapy practice. J Hand Ther 2004;17:165–73.doi:10.1197/j.jht.2004.02.005
        OpenUrlCrossRefPubMed
      21. ↵
        2. Alreni ASE ,
        3. Harrop D ,
        4. Lowe A , et al
        . Measures of upper limb function for people with neck pain. A systematic review of measurement and practical properties. Musculoskelet Sci Pract 2017;29:155–63.doi:10.1016/j.msksp.2017.02.004
        OpenUrl
      22. ↵
        2. Moher D ,
        3. Shamseer L ,
        4. Clarke M , et al
        . Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Syst Rev 2015;4:1.doi:10.1186/2046-4053-4-1
      23. ↵
        2. Law MC ,
        3. MacDermid J
        . Evidence-based rehabilitation : a guide to practice. Thorofare, NJ: Slack Incorporated, 2014.
      24. ↵
        2. Koo TK ,
        3. Li MY
        . A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155–63.doi:10.1016/j.jcm.2016.02.012
        OpenUrlCrossRefPubMed
      25. ↵
        2. Roy J-S ,
        3. Desmeules F ,
        4. MacDermid JC
        . Psychometric properties of presenteeism scales for musculoskeletal disorders: a systematic review. J Rehabil Med 2011;43:23–31.doi:10.2340/16501977-0643
        OpenUrlCrossRefPubMed
      26. ↵
        2. Divaris K ,
        3. Vann WF ,
        4. Baker AD , et al
        . Examining the accuracy of caregivers' assessments of young children's oral health status. J Am Dent Assoc 2012;143:1237–47.doi:10.14219/jada.archive.2012.0071
        OpenUrlAbstract/FREE Full Text
      27. ↵
        2. Cohen J
        . Statistical power analysis for the behavioral sciences, 1988.
      28. ↵
        2. MacDermid JC ,
        3. Walton DM ,
        4. Avery S , et al
        . Measurement properties of the neck disability index: a systematic review. J Orthop Sports Phys Ther 2009;39:400–12.doi:10.2519/jospt.2009.2930
        OpenUrlCrossRefPubMed
      29. ↵
        2. Trippolini MA ,
        3. Dijkstra PU ,
        4. Geertzen JHB , et al
        . Construct validity of functional capacity evaluation in patients with Whiplash-Associated disorders. J Occup Rehabil 2015;25:481–92.doi:10.1007/s10926-014-9555-0
        OpenUrl
      30. ↵
        2. Trippolini MA ,
        3. Dijkstra PU ,
        4. Côté P , et al
        . Can functional capacity tests predict future work capacity in patients with whiplash-associated disorders? Arch Phys Med Rehabil 2014;95:2357–66.doi:10.1016/j.apmr.2014.07.406
        OpenUrl
      31. ↵
        2. Trippolini MA ,
        3. Reneman MF ,
        4. Jansen B , et al
        . Reliability and safety of functional capacity evaluation in patients with whiplash associated disorders. J Occup Rehabil 2013;23:381–90.doi:10.1007/s10926-012-9403-z
        OpenUrl
      32. ↵
        2. Reneman MF ,
        3. Roelofs M ,
        4. Schiphorst Preuper HR
        . Reliability and agreement of neck functional capacity evaluation tests in patients with chronic multifactorial neck pain. Arch Phys Med Rehabil 2017;98:1476–9.doi:10.1016/j.apmr.2016.12.005
        OpenUrl
      33. ↵
        2. van der Meer S ,
        3. Reneman MF ,
        4. Verhoeven J , et al
        . Relationship between self-reported disability and functional capacity in patients with whiplash associated disorder. J Occup Rehabil 2014;24:419–24.doi:10.1007/s10926-013-9473-6
        OpenUrl
      34. ↵
        2. Reesink DD ,
        3. Jorritsma W ,
        4. Reneman MF
        . Basis for a functional capacity evaluation methodology for patients with work-related neck disorders. J Occup Rehabil 2007;17:436–49.doi:10.1007/s10926-007-9086-z
        OpenUrlPubMed
      35. ↵
        2. Lomond KV ,
        3. Côté JN
        . Shoulder functional assessments in persons with chronic neck/shoulder pain and healthy subjects: reliability and effects of movement repetition. Work 2011;38:169–80.doi:10.3233/WOR-2011-1119
        OpenUrlPubMed
      36. ↵
        2. Pierrynowski M ,
        3. McPhee C ,
        4. P Mehta S , et al
        . Intra and inter-rater reliability and convergent validity of FIT-HaNSA in individuals with grade П whiplash associated disorder. Open Orthop J 2016;10:179–89.doi:10.2174/1874325001610010179
        OpenUrl
      37. ↵
        2. Ljungquist T ,
        3. Jensen IB ,
        4. Nygren A , et al
        . Physical performance tests for people with long-term spinal pain: aspects of construct validity. J Rehabil Med 2003;35:69–75.doi:10.1080/16501970306117
        OpenUrlCrossRefPubMedWeb of Science
      38. ↵
        2. Ljungquist T ,
        3. Harms-Ringdahl K ,
        4. Nygren A , et al
        . Intra- and inter-rater reliability of an 11-test package for assessing dysfunction due to back or neck pain. Physiother Res Int 1999;4:214–32.doi:10.1002/pri.167
        OpenUrlCrossRefPubMed
      39. ↵
        2. Ljungquist T ,
        3. Fransson B ,
        4. Harms-Ringdahl K , et al
        . A physiotherapy test package for assessing back and neck dysfunction--discriminative ability for patients versus healthy control subjects. Physiother Res Int : 1999;4:123–40.doi:10.1002/pri.158
        OpenUrlCrossRefPubMed
      40. ↵
        2. Ljungquist T ,
        3. Nygren A ,
        4. Jensen I , et al
        . Physical performance tests for people with spinal pain--sensitivity to change. Disabil Rehabil 2003;25:856–66.doi:10.1080/0963828031000090579
        OpenUrlPubMed
      41. ↵
        2. Reneman MF ,
        3. Dijkstra PU ,
        4. Westmaas M , et al
        . Test-Retest reliability of lifting and carrying in a 2-day functional capacity evaluation. J Occup Rehabil 2002;12:269–75.doi:10.1023/A:1020274624791
        OpenUrlCrossRefPubMedWeb of Science
      42. ↵
        2. Hegedus EJ ,
        3. Vidt ME ,
        4. Tarara DT
        . The best combination of physical performance and self-report measures to capture function in three patient groups. Phys Ther Rev 2014;19:196–203.doi:10.1179/1743288X13Y.0000000121
        OpenUrl
      43. ↵
        2. Roy J-S ,
        3. Macdermid JC ,
        4. Boyd KU , et al
        . Rotational strength, range of motion, and function in people with unaffected shoulders from various stages of life. Sports Med Arthrosc Rehabil Ther Technol 2009;1:4.doi:10.1186/1758-2555-1-4
      44. ↵
        2. Kumta P ,
        3. MacDermid JC ,
        4. Mehta SP , et al
        . The FIT-HaNSA demonstrates reliability and convergent validity of functional performance in patients with shoulder disorders. J Orthop Sports Phys Ther 2012;42:455–64.doi:10.2519/jospt.2012.3796
        OpenUrlPubMed
      45. ↵
        2. MacDermid JC ,
        3. Ghobrial M ,
        4. Quirion KB , et al
        . Validation of a new test that assesses functional performance of the upper extremity and neck (FIT-HaNSA) in patients with shoulder pathology. BMC Musculoskelet Disord 2007;8:42.doi:10.1186/1471-2474-8-42
      46. ↵
        2. Hawkes DH ,
        3. Alizadehkhaiyat O ,
        4. Fisher AC , et al
        . Normal shoulder muscular activation and co-ordination during a shoulder elevation task based on activities of daily living: an electromyographic study. J Orthop Res 2012;30:53–60.doi:10.1002/jor.21482
        OpenUrlPubMed

      Footnotes

      • Twitter @pavlosbob

      • Collaborators CATWAD Co-authors: Michele Sterling m.sterling@uq.edu.au, Anne Söderlund anne.soderlund@mdh.se, Michele Curatolo curatolo@uw.edu, James M Elliott j-elliott@northwestern.edu, David M Walton dwalton5@uwo.ca, Helge Kasch helgkasc@rm.dk, Linda Carroll linda.carroll@ualberta.ca, Hans Westergren Hans.Westergren@skane.se, Samuel A McLean Samuel_McLean@med.unc.edu, Gwendolen Jull g.jull@uq.edu.au, Genevieve Grant genevieve.grant@monash.edu, Luke Connelly l.connelly@uq.edu.au, Joy C MacDermid jmacderm@uwo.ca, Mandy Nielsen mandy.nielsen@griffith.edu.au, Pierre Côté pierre.cote@uoit.ca, Tonny Elmose Andersen tandersen@health.sdu.dk, Trudy Rebbeck trudy.rebbeck@sydney.edu.au, Annick Maujean a.maujean@uq.edu.au, Sarah Robins s.robins1@uq.edu.au, Kenneth Chen k.chen8@uq.edu.au, Julia Treleaven j.treleaven@uq.edu.au

      • Contributors SM contributed significantly to conception and design of the study, data extraction, critical appraisal, interpretation of data and drafting of the manuscript. TS, TA, PB and CC were involved in literature search, critical appraisal and interpretation of data and drafting. AG was involved in critical appraisal and drafting. JM was also involved in the conception and design of the study, drafting and revised the manuscript for important intellectual content. PB and CATWAD were involved in the drafting and review of the manuscript. All authors have given their final approval on the manuscript to be published.

      • Funding This work was supported by the Canadian Institutes of Health Research (CIHR) with funding reference number (FRN: SCA-145102).

      • Competing interests None declared.

      • Patient consent for publication Not required.

      • Provenance and peer review Not commissioned; externally peer reviewed.

      • Data availability statement No data are available.