You are here

  • Home
  • Archive
  • Volume 8, Issue 7
  • Using an interactive virtual environment to integrate a digital Action Research Arm Test, motor imagery and action observation to assess and improve upper limb motor function in patients with neuromuscular impairments: a usability and feasibility study p…

Article Text

Article menu

Download PDFPDF

XML
Rehabilitation medicine
Protocol
Using an interactive virtual environment to integrate a digital Action Research Arm Test, motor imagery and action observation to assess and improve upper limb motor function in patients with neuromuscular impairments: a usability and feasibility study protocol
  1. Frank Behrendt1,2,
  2. Corina Schuster-Amft1,3,4
  1. 1 Research Department, Reha Rheinfelden, Rheinfelden, Switzerland
  2. 2 University Children’s Hospital Basel, Basel, Switzerland
  3. 3 Institute for Rehabilitation and Performance Technology, Bern University of Applied Sciences, Burgdorf, Switzerland
  4. 4 Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland
  1. Correspondence to Dr Frank Behrendt; f.behrendt{at}reha-rhf.ch

Abstract

Introduction In the recent past, training systems using an interactive virtual environment have been introduced to neurorehabilitation. Such systems can be applied to encourage purposeful limb movements and will increasingly be used at home by the individual patient. Therefore, an integrated valid and reliable assessment tool on the basis of such a system to monitor the recovery process would be an essential asset.

Objectives The aim of the study is to evaluate usability, feasibility and validity of the digital version of the Action Research Arm Test using the Bi-Manu-Trainer system as a platform. Additionally, the feasibility and usability of the implementation of action observation and motor imagery tasks into the Bi-Manu-Trainer software will be evaluated.

Patients and methods This observational study is planned as a single-arm trial for testing the new assessment and the action observation and motor imagery training module. Therefore, 75 patients with Parkinson’s disease, multiple sclerosis, stroke, traumatic brain injury or Guillain-Barré syndrome will be included. 30 out of the 75 patients will additionally take part in a 4-week training on the enhanced Bi-Manu-Trainer system. Primary outcomes will be the score on the System Usability Scale and the correlation between the conventional and digital Action Research Arm Test scores. Secondary outcomes will be hand dexterity, upper limb activities of daily living and quality of life.

Hypothesis We hypothesise that the digital Action Research Arm Test assessment is a valid and essential tool and that it is feasible to incorporate action observation and motor imagery into Bi-Manu-Trainer practice. The results are expected to give recommendations for necessary modifications and might also contribute knowledge concerning the application of action observation and motor imagery tasks using a training system such as the Bi-Manu-Trainer.

Trial registration number NCT03268304; Pre-results.

  • rehabilitation medicine
  • stroke
  • parkinson’s disease
  • multiple sclerosis

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/

https://doi.org/10.1136/bmjopen-2017-019646

Statistics from Altmetric.com

    Request Permissions

    If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.

    Introduction 

    Interactive virtual training environments become increasingly used in neurorehabilitation to encourage purposeful limb movements.1 There are several potential advantages: they can enable patients to observe their own movements in real time in the virtual environment. Thereby, patients are able to virtually modify their movements and even to perform related tasks that might hardly be executable in reality. They also provide an opportunity for intensive and varied practice at reduced intervention costs, customised exercise protocols, the ability to monitor the exercise and they can increase user motivation.2 3 In a Cochrane review by Laver et al on the usage of virtual reality (VR) in the field of stroke rehabilitation, it was stated that VR was not superior to conventional therapy approaches in improving upper limb function.4 Nevertheless, it might be beneficial as an additional measure and possibly more effective compared with the same dose of additional conventional therapy.4 In patients with Parkinson’s disease, encouraging findings on the potential benefits have also been found5–8 as well as improvements of balance ability in patients with multiple sclerosis.9 On the contrary, there is only limited evidence on the positive impact of such training systems on motor and cognitive functionality in the rehabilitation of traumatic brain injury. Nevertheless, this approach seems to have the potential to provide a worthwhile therapeutic option also for those patients.10

    The Bi-Manu-Trainer (BMT) (Reha Stim Medtec AG) is an example of a therapeutic VR system for upper limb training in rehabilitation (figure 1). It provides three feedback modalities (acoustic, visual and sensory) to facilitate performance adaption and offers the opportunity to perform unimanual or bimanual movement tasks of the upper limb with different game options on a computer screen. The device was developed specifically for arm, hand and finger movements only. It allows detecting and displaying even small movement changes.11 It has recently been shown that BMT training and conventional training both seem to have a similar effect on hand dexterity in patients with chronic stroke over a 4-week training period including 16 training sessions.12

    Figure 1
    Figure 1

    Virtual reality training system setup (Bi-Manu-Trainer, Reha Stim Medtec AG). The model wears wireless hand gloves with movement sensors attached. The screen displays real-time hand and finger positions.

    However, the BMT system is being continuously revised and its newly developed features are the basis for the planned clinical study. The overall aim is therefore to evaluate the integration of two new software modules. One module was planned to provide the possibility of assessing upper limb motor function using the BMT system (project 1) and the other module integrates action observation (AO) and motor imagery (MI) training into the BMT training (project 2). We hypothesise that the digital Action Research Arm Test (d-ARAT) is a valid and useful assessment tool, and that it is furthermore feasible to incorporate AO and MI into the BMT system.

    Project 1: integration of the ARAT into the BMT system

    A valid assessment of the upper limb functioning, which can be carried out by the patient himself, might possibly be an appreciable improvement of the BMT training system, especially when applied at home. This could provide both more frequent information about the individual progress and also allow for an automatic individual adjustment of the training software, that is, the level of difficulty. The conventional ARAT13 requires a human examiner to transform observations of a patient’s performance into a score in order to set according treatment goals and select appropriate treatment methods. However, this process is quite time consuming. The new assessment module for the BMT on the basis of the conventional ARAT is envisaged to provide the possibility of self-administering the test with or without examiner supervision. The BMT hardware also comprises wearable sensor technology that can be used to measure motor abilities or to monitor rehabilitation outcome for instance by estimating Fugl-Meyer clinical scores based on motion data.14–16 Accordingly, the novel assessment module was developed to be capable of processing the recorded hand and finger movement data and thereby to judge the movement performance. In this regard, the assessment module for the BMT system was designed to incorporate most of the test items of the established ARAT.

    Project 2: AO and MI as integral part of the BMT training

    It is now accepted in neurophysiology that the observation of actions performed by others can activate some of the neural structures also responsible for the actual execution of the same actions as there is an overlap of the visual and the motor system.17 Interestingly, during both the actual execution and observation, an increase of force in performing the same movement was found in both hands when compared with a control condition.18 In the field of rehabilitation, AO has been shown to facilitate motor learning and the building of a motor memory trace in normal adults as well as in patients with stroke.19 20

    Similarly, MI is a dynamic state during which the representation of a specific motor action is internally activated without any motor output.21 MI requires the conscious activation of brain regions that are also involved in movement preparation and execution, accompanied by a voluntary inhibition of the actual movement.22

    With respect to MI, efficacy has frequently been proved for patients with poststroke23 24 with positive effects on upper extremity motor recovery, balance and gait in patients with stroke.25–27

    As being valuable methods in rehabilitation, both AO and MI applied in combination can be even more effective concerning the cortical activation pattern and the corticospinal excitability, respectively.28 29 Project 2 of the planned clinical trial, therefore, aims at the implementation of AO and MI tasks into the BMT training.

    Primary and secondary objectives

    Primary objectives

    The overall aim of the study is to evaluate the two novel modules within the BMT system applied in clinical practice. Therefore, the study will investigate the usability of both modules (see table 1 for an overview). It is further planned to evaluate the validity of the new assessment module (project 1), that is, currently being developed on the basis of the conventional ARAT.

    View this table:
    Table 1

    Overview of study objectives

    Secondary objectives

    The secondary objective of project 1 is the evaluation of reliability and responsiveness of the assessment module, whereas the secondary objective of project 2 is the evaluation of feasibility and applicability of the AO and MI module. The different feasibility parameters are thought to provide details for an ensuing randomised controlled trial such as most importantly information to enable a sample size calculation.

    Methods and design

    The study is designed as a single-arm trial. All study parts (see figure 2 for an overview) will conform the guidelines of good clinical practice and the Declaration of Helsinki. Data collection will be performed in a rehabilitation clinic in the German-speaking part of Switzerland.

    Figure 2
    Figure 2

    Study overview: T0, pre-training; T1, measurement after eight training sessions; T2, measurement after eight further training sessions; FU, measurement after 2-month follow-up period. Project 1—Integration of the Action Research Arm Test into the BMT system; Project 2—action observation and motor imagery as integral part of the BMT training. BMT, Bi-Manu-Trainer; FU, follow-up.

    Patient selection criteria and recruitment

    Patients will be eligible for study participation if they fulfil the selection criteria listed in table 2. They equally apply for both projects. The patient recruitment strategy employs different approaches:

    1. Patients will be recruited from the clinics’ inpatient or outpatient departments by physicians, therapists and nurses.

    2. Patients will be recruited from the clinics’ patient database. Datasets will be screened for study selection criteria by the involved study personnel. If patients are eligible, they will be sent a letter describing the study and including patient information. If patients are interested in participating, they can contact the study personnel in the clinic by phone or email.

    3. Patients will be recruited via a study information flyer provided on the clinic’s homepage and through patient self-help groups. If patients are interested in participating, they can contact the study personnel in the responsible clinic by phone or email.

    View this table:
    Table 2

    Patient selection criteria

    Written informed consent will be obtained from all patients after they received written and oral study information. The procedure will be performed by the study personnel before patient inclusion. Patient information and consent forms both in German can be obtained from the first author. Interested patients will have the choice to either take part solely in the evaluation of the new assessment module (project 1) or both modules (projects 1 and 2, see figure 2). Participation in project 2 includes therapy sessions with the new AO-MI module.

    Study procedure

    At the first contact, the patient will be informed and invited to partake. If willingness is confirmed within 1–2 days, an appointment for eligibility evaluation and a baseline assessment is made. At the first visit, a baseline assessment shall confirm eligibility according to MMSE, Chedoke Arm and Hand Activity Inventory (CAHAI) and Box and Block Test (BBT) scores. All participating patients will receive their standard therapies during the study as before. The chronological order of all assessments and practice sessions including all outcome measures can be found in table 3.

    View this table:
    Table 3

    Study procedures

    The additional practice with the new AO-MI module in project 2 is described in table 4 using the Template for Intervention Description and Replication checklist and guide.30

    View this table:
    Table 4

    Description of the action observation and motor imagery (AO-MI) practice

    Outcome parameters

    All data will be collected on a case report form that will be stored in a locked cabinet and will not be accessible for treating therapists. The standardised case report form can also be obtained from the first author (FB).

    Any patient who decides to stop participation in the training will still be invited to all further scheduled measurement events so that the recovery process can be further assessed. See table 5 for an overview of all assessments and the related measurement events.

    View this table:
    Table 5

    Overview of outcome measures

    Primary outcomes

    Action Research Arm Test

    The primary outcome of interest is the correlation between the scores achieved on the conventional ARAT13 and the d-ARAT. The ARAT, first described by Lyle,31 evaluates 19 tests of arm motor function, both distally and proximally. It is an evaluative measure to assess specific changes in upper limb function among individuals who sustained cortical damage resulting in hemiplegia.31 It is a reliable, valid measure of arm motor status and a valuable tool for characterising clinical state and for assessing spontaneous and therapy-induced recovery.13 The ARAT assesses the patient’s ability to handle objects differing in size, weight and shape and is therefore a valuable arm-specific measure of activity limitation.32 Like other motor assessments, it needs an examiner to transfer patient’s movements into a score.13 With the new digital ARAT, the patient’s upper limb movements will automatically be analysed and rated. The correlation values will be used to determine the validity, test–retest reliability and responsiveness of the d-ARAT.

    System Usability Scale

    The System Usability Scale (SUS) provides a quick and reliable tool for measuring the perceived usability of any system,33 and is a simple, 10-item attitude Likert scale giving a global view of subjective assessments of usability. It consists of a questionnaire with five response options for respondents; from strongly agree to strongly disagree.34 It can be used on small sample sizes with reliable results and can effectively differentiate between usable and unusable systems.33 35 Furthermore, it is quick and easy for study participants to complete and for administrators to score.36 In the described study, the SUS will be deployed to assess the patients’ judgement of the usability of both new BMT modules.

    Secondary outcomes

    Box and Block Test

    Change in hand dexterity is one of the secondary outcomes of interest. Numerous tests for manual dexterity have been developed, for instance, the BBT.37 38 The BBT is a quick, simple and reliable measurement of manual dexterity and its administration procedure is standardised.39 It consists of moving the maximum number of blocks one by one from one compartment of a box to another of equal size within 1 min.39 It is often assessed in rehabilitation since it is an essential feature of upper limp.39 The BBT has been used in patients after stroke, in patients with multiple sclerosis and traumatic brain injury.40 It is a reliable and valid assessment tool, provides normative data for healthy adult individuals and is frequently used in research and rehabilitation of both children and adults.41

    Chedoke Arm and Hand Activity Inventory

    Upper limb activities of daily living (ADL) function is planned be assessed using the CAHAI V.9. The CAHAI was introduced to include relevant functional tasks and to be sensitive to clinically important changes in upper limb function.42 It is a validated upper limb measure that uses a seven-point quantitative scale in order to assess functional recovery of the arm and hand.43 The CAHAI scores represent the patients relative ability to independently perform stabilisation or manipulation in ADL with the affected upper limb.44 A score of 1 represents total dependence on another person, and a score of 7 indicates patient’s independence without time or safety concerns or necessary splints or devices. A high interrater reliability and convergent and discriminant cross-sectional validity were established for the CAHAI42 45 and it was found to be more sensitive to clinically important change than the ARAT.42

    EuroQol 5-Dimension Questionnaire with 5-level Scale

    Participants’ quality of life will be assessed using the EuroQol 5-Dimension Questionnaire with 5-level Scale (EQ-5D-5L) questionnaire.46 47 The EQ-5D is an instrument for the evaluation of quality of live.48 It is based on a descriptive system that defines health in terms of five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression and was designed to measure decrements in health.48 The EQ-5D is a standardised instrument for measuring the generic health status and was first introduced in 1990.46 It has been demonstrated that it is reproducible and valid on the evaluation of quality of life in patients with post-stroke.49 The questionnaire is already translated into several languages, is frequently used and has shown internal consistency when applied to a general population and to groups of patients with various diseases.50

    Patient Global Impression of Change

    The self-report measure Patient Global Impression of Change (PGIC) reflects a patient’s belief about the efficacy of treatment.51 It is widely used and aims at evaluating all aspects of patients’ health and determining if there has been an improvement or not. PGIC ratings are increasingly being used as ‘gold standard’ in order to assess clinically important change in different conditions.52–54 The patient has to select the one response from the response options on a seven-point scale that gives the most accurate description of the state of health which thus reveals the patient’s rating of the overall improvement.51 55 Possible ratings are ‘very much improved’, ‘much improved’, ‘minimally improved’, ‘no change’, ‘minimally worse’, ‘much worse’ or ‘very much worse’.

    Determination of sample size

    Project 1

    In order to determine an adequate sample size for testing the psychometrics, we assessed the available literature on the ARAT. It has been reported to be valid,13 32 56–58 reliable13 31 32 57 and sensitive to change.59–61 The number of included patients for the estimation of the validity coefficients varies between the cited publications from 12 to 59. However, Hobart et al found that validity estimates would generally be stable in samples n≥80, for 75% of scales in samples of 40 subjects and for 50% of scales in samples of 20.62 They also stated that sample sizes of a minimum of 20 for reliability provided highly representative estimates.62 Thus, after consideration of the available literature and project constraints, we decided to include 75 patients in order to check for the validity of the d-ARAT module and the other objectives of project 1 (table 1). Of these 75 patients, 30 are planned to additionally practise on the AO-MI module in project 2.

    Project 2

    Further, as it is not intended in the course of this study to apply confirmatory statistics for the evaluation of the AO-MI module a sample size calculation by conducting a power analysis is also not envisaged for project 2. Substantial and reliable data on the effect of BMT practice in adults have not yet been published. Therefore, the sample size determination for project 2 had to rely on according available literature on pilot and feasibility study methodology.63–70 We decided on that basis to include 30 patients into project 2 which aims to evaluate the practice on the AO-MI module.

    Statistical analysis

    Primary study objectives

    A core part of the study is to assess whether the d-ARAT is a valid measurement tool and whether it thereby provides the possibility of accurately evaluating upper limb motor function. As mentioned above, the ARAT has repeatedly been validated before by comparing it to different assessments like the Fugl-Meyer Stroke Assessment.13 Thus, we decided to assess the concurrent validity of the new d-ARAT by comparing its score to the score of the conventional ARAT which will also be assessed at the same measurement events. Validity measures will be evaluated using the Pearson’s correlation coefficient (r)13 which will be considered to indicate high correlation between the conventional ARAT and the d-ARAT and thus a good concurrent validity in case it is greater than 0.75.71 With regard to the evaluation of the usability of both modules, the analysis of the SUS data and the categorisation of the results will be performed on the basis of descriptive statistics.

    Secondary study objectives

    The intraclass correlation coefficient (ICC) is planned to be used to evaluate test–retest reliability of the d-ARAT assessment module. The ICC has been widely used in conservative care medicine to evaluate interrater, test–retest and intrarater reliability.72–78 For test–retest reliability studies, it was proposed to choose a two-way mixed-effects model along with an absolute agreement definition.79 The latter form of ICC will accordingly be used in this study based on the mean of several measurements. Furthermore, two different approaches of responsiveness recommended by Crosby et al will be applied: Criterion-referenced change (or anchor-based methods) and precision-referenced change (or distribution-based methods).80 For the criterion-referenced change, the d-ARAT change scores will be compared with the BBT change scores using Pearson’s correlation coefficient (r). The precision-referenced change will be analysed by calculating the minimal detectable change. The standardised response mean will be calculated as the ratio of observed change and its SD, which serves as an indicator for the magnitude of change.80

    The feasibility of implementing the described practice using the AO-MI module into the BMT training and the use of the digital assessment will be reported using descriptive statistics. This will comprise several outcomes.

    1. Recruitment rate: It will be considered good if more than 85% of the patients addressed partake in the trial.

    2. Eligibility criteria: The applied criteria can be seen as adequate when the selected patients are able to partake in the assessments and also in the intervention if applicable. It is necessary to evaluate whether the criteria a too inclusive or restrictive.

    3. Drop-out rate: A number of less than 15% of the patients leaving the trial will be regarded as good.

    4. Adherence: A participation in at least 80% of the practice sessions will be considered good. As soon as a patient misses one-third of it the respective data will be analysed separately.

    5. The treatment effect will be calculated using Cohen’s d which is the standardised mean difference in order to perform a sample size determination for a randomised controlled trial.

    Dissemination policy

    The study personnel will adhere to an open access policy:

    • The trialists intend to publish the study protocol and the study results in international open access journals to provide easy access to the study documents for all interested readers.

    • The study is registered in an international open access clinical trial database (ClinicalTrials.gov Identifier: NCT03268304).

    • As the study progresses, its methods and preliminary results will be presented at national and international congresses and workshops.

    • After study finalisation and data analyses, all study patients will receive a plain language summary of the study results.

    Involvement of professional writers is not intended. No restrictions will be placed on the publication of positive or negative results. Though it is actually not a randomised trial, the study results will be reported in accordance with the guidelines set forth in the 2010 Consolidated Standards of Reporting Trials (CONSORT).81

    Criteria for halting the trial

    At present, the commercial BMT system has been used for more than 5 years with numerous patients (children, adults) in different acute hospitals and rehabilitation clinics. So far, no adverse events have been reported. However, this study will be halted if any of the following criteria are fulfilled:

    • More than three patients report a sudden onset of or increase in shoulder pain during or just after therapy that is highly likely to be attributable to the use of the BMT, and which does not immediately cease after stopping.

    • More than 25% of the patients report severe cybersickness during BMT training which persists after training is halted.

    • Epileptic seizures in at least two patients are induced directly while using BMT.

    Patients reporting the criteria mentioned above will be evaluated by the physician on duty and will be assessed and followed up for the originally planned study duration.

    Patient involvement

    Patients were not involved in the design of this study.

    Discussion

    The aim of the described study is to examine two newly developed software modules for the BMT training device which is already being used in clinics and rehabilitation centres. First, the new d-ARAT assessment module developed on the basis of the established ARAT will be evaluated with respect to its validity, usability, reliability and responsiveness (project 1). Second, the other module which is planned to integrate AO and MI into the BMT system will be checked for its usability, applicability and its feasibility (project 2). Both modules are intended to enhance the BMT system in terms of its functionality and usability also with regard to an individual home use. The individual use could be a small part of the answer of how to deal with the possibly growing number of patients in the future who may need neurorehabilitation training especially when individual care cannot be provided in sufficient quantity and quality. Changing living and working conditions and an improved health-related behaviour are as well leading to an increase in the portion of the population accounted for by the elderly. Therefore, the number of people with disabilities or chronic diseases is constantly rising and the consequences of an altering age structure need to be coped with in particular in the health sector. The rehabilitation technology industry is rapidly evolving and the training systems are constantly enhanced to gain an increased efficacy and usability, and to get well applicable also in home use. The currently planned developments for the BMT system are in line with that process. The results are intended to be used for the evaluation and for supporting the further development and optimisation process of the software and therefore for the patients’ use. The trial will also presumably contribute knowledge concerning the application of relatively new rehabilitation methods such as AO and MI tasks within rehabilitation training using a virtual environment. It is further intended to provide hints on the feasibility and the limitations of a digital motor assessment module. It can be assumed that such a system entails advantages and disadvantages. Obviously, the software algorithm for judging the different movement tasks will be completely objective which prevents a variability of the achieved scores due to interindividual and intraindividual differences of the judging person. On the other hand, the level of precision of the sensors might have an influence on the validity of the new module. However, the successful completion of the planned software development could possibly enrich the range of available assessment and rehabilitation options.

    Dissemination

    All dissemination will be undertaken using the CONSORT Statement recommendations. Results will be published in peer-reviewed journals and at conference presentations.

    References

    1. 1.↵
      2. Schuster-Amft C ,
      3. Eng K ,
      4. Lehmann I , et al
      . Using mixed methods to evaluate efficacy and user expectations of a virtual reality-based training system for upper-limb recovery in patients after stroke: a study protocol for a randomised controlled trial. Trials 2014;15:1–11.doi:10.1186/1745-6215-15-350
      OpenUrlCrossRefPubMed
    2. 2.↵
      2. Lewis GN ,
      3. Woods C ,
      4. Rosie JA ,
      5. a RJ , et al
      . Virtual reality games for rehabilitation of people with stroke: perspectives from the users. Disabil Rehabil Assist Technol 2011;6:453–63.doi:10.3109/17483107.2011.574310
      OpenUrlCrossRefPubMed
    3. 3.↵
      2. Merians AS ,
      3. Fluet GG ,
      4. Qiu Q , et al
      . Learning in a virtual environment using haptic systems for movement re-education: can this medium be used for remodeling other behaviors and actions? J Diabetes Sci Technol 2011;5:301–8.doi:10.1177/193229681100500215
      OpenUrlCrossRefPubMed
    4. 4.↵
      2. Laver KE ,
      3. Lange B ,
      4. George S , et al
      . Virtual reality for stroke rehabilitation. Cochrane Database Syst Rev 2017;11:CD008349.doi:10.1002/14651858.CD008349.pub4
      OpenUrl
    5. 5.↵
      2. Mirelman A ,
      3. Maidan I ,
      4. Herman T , et al
      . Virtual reality for gait training: can it induce motor learning to enhance complex walking and reduce fall risk in patients with Parkinson’s disease? J Gerontol A Biol Sci Med Sci 2011;66:234–40.doi:10.1093/gerona/glq201
      OpenUrlPubMedWeb of Science
    6. 6.↵
      2. Mirelman A ,
      3. Maidan I ,
      4. Deutsch JE
      . Virtual reality and motor imagery: promising tools for assessment and therapy in Parkinson’s disease. Mov Disord 2013;28:1597–608.doi:10.1002/mds.25670
      OpenUrlCrossRefPubMed
    7. 7.↵
      2. Badarny S ,
      3. Aharon-Peretz J ,
      4. Susel Z , et al
      . Virtual reality feedback cues for improvement of gait in patients with Parkinson’s disease. Tremor Other Hyperkinet Mov 2014;4:225.doi:10.7916/D8V69GM4
      OpenUrl
    8. 8.↵
      2. Song GB ,
      3. Park EC
      . Effect of virtual reality games on stroke patients' balance, gait, depression, and interpersonal relationships. J Phys Ther Sci 2015;27:2057–60.doi:10.1589/jpts.27.2057
      OpenUrl
    9. 9.↵
      2. Eftekharsadat B ,
      3. Babaei-Ghazani A ,
      4. Mohammadzadeh M , et al
      . Effect of virtual reality-based balance training in multiple sclerosis. Neurol Res 2015;37:539–44.doi:10.1179/1743132815Y.0000000013
      OpenUrlCrossRefPubMed
    10. 10.↵
      2. Pietrzak E ,
      3. Pullman S ,
      4. McGuire A
      . Using virtual reality and videogames for traumatic brain injury rehabilitation: a structured literature review. Games Health J 2014;3:202–14.doi:10.1089/g4h.2014.0013
      OpenUrl
    11. 11.↵
      2. Eng K ,
      3. Siekierka E ,
      4. Pyk P , et al
      . Interactive visuo-motor therapy system for stroke rehabilitation. Med Biol Eng Comput 2007;45:901–7.doi:10.1007/s11517-007-0239-1
      OpenUrlCrossRefPubMedWeb of Science
    12. 12.↵
      2. Schuster-Amft C ,
      3. Eng K ,
      4. Suica Z , et al
      . Effect of a four-week virtual reality-based training versus conventional therapy on upper limb motor function after stroke: a multicenter parallel group randomized trial. 2018. Manuscript.
    13. 13.↵
      2. Yozbatiran N ,
      3. Der-Yeghiaian L ,
      4. Cramer SC
      . A standardized approach to performing the action research arm test. Neurorehabil Neural Repair 2008;22:78–90.doi:10.1177/1545968307305353
      OpenUrlCrossRefPubMedWeb of Science
    14. 14.↵
      2. Hester T ,
      3. Hughes R ,
      4. Sherrill DM , et al
      . Using wearable sensors to measure motor abilities following stroke. Wearable implant body sens networks, 2006 BSN 2006 int work. Cambridge, MA, USA: IEEE, 2006:1–4.
    15. 15.↵
      2. Patel S ,
      3. Hughes R ,
      4. Hester T , et al
      . A novel approach to monitor rehabilitation outcomes in stroke survivors using wearable technology. Proc IEEE Inst Electr Electron Eng 2010;98:450–61.doi:10.1109/JPROC.2009.2038727
      OpenUrl
    16. 16.↵
      2. Del Din S ,
      3. Patel S ,
      4. Cobelli C , et al
      . Estimating Fugl-Meyer clinical scores in stroke survivors using wearable sensors. Conf Proc IEEE Eng Med Biol Soc 2011;2011:5839–42.doi:10.1109/IEMBS.2011.6091444
      OpenUrlPubMed
    17. 17.↵
      2. Fabbri-Destro M ,
      3. Rizzolatti G
      . Mirror neurons and mirror systems in monkeys and humans. Physiology 2008;23:171–9.doi:10.1152/physiol.00004.2008
      OpenUrlCrossRefPubMed
    18. 18.↵
      2. Porro CA ,
      3. Facchin P ,
      4. Fusi S , et al
      . Enhancement of force after action observation: behavioural and neurophysiological studies. Neuropsychologia 2007;45:3114–21.doi:10.1016/j.neuropsychologia.2007.06.016
      OpenUrlCrossRefPubMedWeb of Science
    19. 19.↵
      2. Stefan K ,
      3. Cohen LG ,
      4. Duque J , et al
      . Formation of a motor memory by action observation. J Neurosci 2005;25:9339–46.doi:10.1523/JNEUROSCI.2282-05.2005
      OpenUrlAbstract/FREE Full Text
    20. 20.↵
      2. Celnik P ,
      3. Webster B ,
      4. Glasser DM , et al
      . Effects of action observation on physical training after stroke. Stroke 2008;39:1814–20.doi:10.1161/STROKEAHA.107.508184
      OpenUrlAbstract/FREE Full Text
    21. 21.↵
      2. Decety J ,
      3. Perani D ,
      4. Jeannerod M , et al
      . Mapping motor representations with positron emission tomography. Nature 1994;371:600–2.doi:10.1038/371600a0
      OpenUrlCrossRefPubMedWeb of Science
    22. 22.↵
      2. Lotze M ,
      3. Cohen LG
      . Volition and imagery in neurorehabilitation. Cogn Behav Neurol 2006;19:135–40.doi:10.1097/01.wnn.0000209875.56060.06
      OpenUrlCrossRefPubMedWeb of Science
    23. 23.↵
      2. Ietswaart M ,
      3. Johnston M ,
      4. Dijkerman HC , et al
      . Mental practice with motor imagery in stroke recovery: randomized controlled trial of efficacy. Brain 2011;134(Pt 5):1373–86.doi:10.1093/brain/awr077
      OpenUrlCrossRefPubMedWeb of Science
    24. 24.↵
      2. Braun S ,
      3. Kleynen M ,
      4. van Heel T , et al
      . The effects of mental practice in neurological rehabilitation; a systematic review and meta-analysis. Front Hum Neurosci 2013;7:1–23.doi:10.3389/fnhum.2013.00390
      OpenUrlCrossRefPubMed
    25. 25.↵
      2. Cho HY ,
      3. Kim JS ,
      4. Lee GC
      . Effects of motor imagery training on balance and gait abilities in post-stroke patients: a randomized controlled trial. Clin Rehabil 2013;27:675–80.doi:10.1177/0269215512464702
      OpenUrlCrossRefPubMed
    26. 26.↵
      2. Hosseini SA ,
      3. Fallahpour M ,
      4. Sayadi M , et al
      . The impact of mental practice on stroke patients' postural balance. J Neurol Sci 2012;322:263–7.doi:10.1016/j.jns.2012.07.030
      OpenUrlCrossRefPubMed
    27. 27.↵
      2. Dijkerman HC ,
      3. Ietswaart M ,
      4. Johnston M , et al
      . Does motor imagery training improve hand function in chronic stroke patients? A pilot study. Clin Rehabil 2004;18:538–49.doi:10.1191/0269215504cr769oa
      OpenUrlCrossRefPubMedWeb of Science
    28. 28.↵
      2. Sakamoto M ,
      3. Muraoka T ,
      4. Mizuguchi N , et al
      . Combining observation and imagery of an action enhances human corticospinal excitability. Neurosci Res 2009;65:23–7.doi:10.1016/j.neures.2009.05.003
      OpenUrlCrossRefPubMedWeb of Science
    29. 29.↵
      2. Villiger M ,
      3. Estévez N ,
      4. Hepp-Reymond MC , et al
      . Enhanced activation of motor execution networks using action observation combined with imagination of lower limb movements. PLoS One 2013;8:e72403.doi:10.1371/journal.pone.0072403
    30. 30.↵
      2. Hoffmann TC ,
      3. Glasziou PP ,
      4. Boutron I , et al
      . Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ 2014;348:g1687.doi:10.1136/bmj.g1687
      OpenUrlAbstract/FREE Full Text
    31. 31.↵
      2. Lyle RC
      . A performance test for assessment of upper limb function in physical rehabilitation treatment and research. Int J Rehabil Res 1981;4:483–92.doi:10.1097/00004356-198112000-00001
      OpenUrlCrossRefPubMedWeb of Science
    32. 32.↵
      2. Platz T ,
      3. Pinkowski C ,
      4. van Wijck F , et al
      . Reliability and validity of arm function assessment with standardized guidelines for the fugl-meyer test, action research arm test and box and block test: a multicentre study. Clin Rehabil 2005;19:404–11.doi:10.1191/0269215505cr832oa
      OpenUrlCrossRefPubMedWeb of Science
    33. 33.↵
      2. Brooke J
      . SUS - A quick and dirty usability scale. Usability Eval Ind 1996;189:4–7.
      OpenUrl
    34. 34.↵
      2. Sauro J
      . Measuring Usability With The System Usability Scale (SUS). Meas Usability 2011:1–5.
    35. 35.↵
      2. Peres SC ,
      3. Pham T ,
      4. Phillips R
      . Validation of the system usability scale (sus): Sus in the wild. Proc Hum Factors Ergon Soc 2013;1:192–6.
      OpenUrl
    36. 36.↵
      2. Bangor A ,
      3. Kortum P ,
      4. Miller J
      . Determining what individual SUS scores mean: adding an adjective rating scale. J usability Stud 2009;4:114–23.
      OpenUrl
    37. 37.↵
      2. Cromwell F
      . Occupational therapist manual for basic skills assessment: primary prevocational evaluation. Pasadena: Fair Oaks Prining, 1965:29–31.
    38. 38.↵
      2. Williams ME ,
      3. Hadler NM ,
      4. Earp JA
      . Manual ability as a marker of dependency in geriatric women. J Chronic Dis 1982;35:115–22.doi:10.1016/0021-9681(82)90112-6
      OpenUrlCrossRefPubMedWeb of Science
    39. 39.↵
      2. Desrosiers J ,
      3. Bravo G ,
      4. Hébert R , et al
      . Validation of the box and block test as a measure of dexterity of elderly people: reliability, validity, and norms studies. Arch Phys Med Rehabil 1994;75:751–5.
      OpenUrlPubMedWeb of Science
    40. 40.↵
      2. Connell LA ,
      3. Tyson SF
      . Clinical reality of measuring upper-limb ability in neurologic conditions: a systematic review. Arch Phys Med Rehabil 2012;93:221–8.doi:10.1016/j.apmr.2011.09.015
      OpenUrlCrossRefPubMed
    41. 41.↵
      2. Mathiowetz V ,
      3. Volland G ,
      4. Kashman N , et al
      . Adult norms for the box and block test of manual dexterity. Am J Occup Ther 1985;39:386–91.doi:10.5014/ajot.39.6.386
      OpenUrlCrossRefPubMedWeb of Science
    42. 42.↵
      2. Barreca SR ,
      3. Stratford PW ,
      4. Lambert CL , et al
      . Test-retest reliability, validity, and sensitivity of the chedoke arm and hand activity inventory: a new measure of upper-limb function for survivors of stroke. Arch Phys Med Rehabil 2005;86:1616–22.doi:10.1016/j.apmr.2005.03.017
      OpenUrlCrossRefPubMedWeb of Science
    43. 43.↵
      2. Barreca SR ,
      3. Stratford PW ,
      4. Masters LM , et al
      . Comparing 2 versions of the chedoke arm and hand activity inventory with the action research arm test. Phys Ther 2006;86:245–53.
      OpenUrlAbstract/FREE Full Text
    44. 44.↵
      2. Barreca S ,
      3. Gowland CK ,
      4. Stratford P , et al
      . Development of the chedoke arm and hand activity inventory: theoretical constructs, item generation, and selection. Top Stroke Rehabil 2004;11:31–42.doi:10.1310/JU8P-UVK6-68VW-CF3W
      OpenUrlCrossRefPubMed
    45. 45.↵
      2. Schuster C ,
      3. Hahn S ,
      4. Ettlin T
      . Objectively-assessed outcome measures: a translation and cross-cultural adaptation procedure applied to the Chedoke McMaster Arm and Hand Activity Inventory (CAHAI). BMC Med Res Methodol 2010;10:106.doi:10.1186/1471-2288-10-106
      OpenUrlCrossRefPubMed
    46. 46.↵
      Group E. EuroQol-a new facility for the measurement of health-related quality of life. Health Policy 1990;16:199–208.
      OpenUrlCrossRefPubMedWeb of Science
    47. 47.↵
      2. Herdman M ,
      3. Gudex C ,
      4. Lloyd A , et al
      . Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011;20:1727–36.doi:10.1007/s11136-011-9903-x
      OpenUrlCrossRefPubMedWeb of Science
    48. 48.↵
      2. Brooks R
      . EuroQol: the current state of play. Health Policy 1996;37:53–72.doi:10.1016/0168-8510(96)00822-6
      OpenUrlCrossRefPubMedWeb of Science
    49. 49.↵
      2. Pinto EB ,
      3. Maso I ,
      4. Vilela RN , et al
      . Validation of the EuroQol quality of life questionnaire on stroke victims. Arq Neuropsiquiatr 2011;69:320–3.doi:10.1590/S0004-282X2011000300010
      OpenUrlCrossRefPubMed
    50. 50.↵
      2. Wolfs CA ,
      3. Dirksen CD ,
      4. Kessels A , et al
      . Performance of the EQ-5D and the EQ-5D+C in elderly patients with cognitive impairments. Health Qual Life Outcomes 2007;5:33.doi:10.1186/1477-7525-5-33
      OpenUrlCrossRefPubMed
    51. 51.↵
      2. Ferguson L ,
      3. Scheman J
      . Patient global impression of change scores within the context of a chronic pain rehabilitation program. The Journal of Pain 2009;10:S73.doi:10.1016/j.jpain.2009.01.258
      OpenUrl
    52. 52.↵
      2. Geisser ME ,
      3. Robinson ME ,
      4. Keefe FJ , et al
      . Catastrophizing, depression and the sensory, affective and evaluative aspects of chronic pain. Pain 1994;59:79–83.doi:10.1016/0304-3959(94)90050-7
      OpenUrlCrossRefPubMedWeb of Science
    53. 53.↵
      2. Farrar JT ,
      3. Young JP ,
      4. LaMoreaux L , et al
      . Clinical importance of changes in chronic pain intensity measured on an 11-point numerical pain rating scale. Pain 2001;94:149–58.doi:10.1016/S0304-3959(01)00349-9
      OpenUrlCrossRefPubMedWeb of Science
    54. 54.↵
      2. Salaffi F ,
      3. Stancati A ,
      4. Silvestri CA , et al
      . Minimal clinically important changes in chronic musculoskeletal pain intensity measured on a numerical rating scale. Eur J Pain 2004;8:283–91.doi:10.1016/j.ejpain.2003.09.004
      OpenUrlCrossRefPubMedWeb of Science
    55. 55.↵
      2. Guy W
      . ECDEU assessment manual for psychopharmacology. revised. Washington, DC: US Department of Health, Education, and Welfare, 1976.
    56. 56.↵
      2. Lin KC ,
      3. Chuang LL ,
      4. Wu CY , et al
      . Responsiveness and validity of three dexterous function measures in stroke rehabilitation. J Rehabil Res Dev 2010;47:563–71.doi:10.1682/JRRD.2009.09.0155
      OpenUrlCrossRefPubMed
    57. 57.↵
      2. Hsieh CL ,
      3. Hsueh IP ,
      4. Chiang FM , et al
      . Inter-rater reliability and validity of the action research arm test in stroke patients. Age Ageing 1998;27:107–13.doi:10.1093/ageing/27.2.107
      OpenUrlCrossRefPubMedWeb of Science
    58. 58.↵
      2. Hsieh YW ,
      3. Wu CY ,
      4. Lin KC , et al
      . Responsiveness and validity of three outcome measures of motor function after stroke rehabilitation. Stroke 2009;40:1386–91.doi:10.1161/STROKEAHA.108.530584
      OpenUrlAbstract/FREE Full Text
    59. 59.↵
      2. Hsueh IP ,
      3. Hsieh CL
      . Responsiveness of two upper extremity function instruments for stroke inpatients receiving rehabilitation. Clin Rehabil 2002;16:617–24.doi:10.1191/0269215502cr530oa
      OpenUrlCrossRefPubMedWeb of Science
    60. 60.↵
      2. De Weerdt WJG ,
      3. Harrison MA
      . Measuring recovery of arm-hand function in stroke patients: a comparison of the brunnstrom-fugl-meyer test and the action research arm test. Physiotherapy Canada 1985;37:65–70.doi:10.3138/ptc.37.2.065
      OpenUrlCrossRef
    61. 61.↵
      2. Page SJ ,
      3. Levine P ,
      4. Leonard AC
      . Modified constraint-induced therapy in acute stroke: a randomized controlled pilot study. Neurorehabil Neural Repair 2005;19:27–32.doi:10.1177/1545968304272701
      OpenUrlCrossRefPubMedWeb of Science
    62. 62.↵
      2. Hobart JC ,
      3. Cano SJ ,
      4. Warner TT , et al
      . What sample sizes for reliability and validity studies in neurology? J Neurol 2012;259:2681–94.doi:10.1007/s00415-012-6570-y
      OpenUrlCrossRefPubMedWeb of Science
    63. 63.↵
      2. Nasreddine ZS ,
      3. Phillips NA ,
      4. Bédirian V , et al
      . The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005;53:695–9.doi:10.1111/j.1532-5415.2005.53221.x
      OpenUrlCrossRefPubMedWeb of Science
    64. 64.↵
      2. Connelly LM
      . Pilot studies. Medsurg Nurs 2008;17:411–2.
      OpenUrlPubMed
    65. 65.↵
      2. Hertzog MA
      . Considerations in determining sample size for pilot studies. Res Nurs Health 2008;31:180–91.doi:10.1002/nur.20247
      OpenUrlCrossRefPubMedWeb of Science
    66. 66.↵
      2. Julious SA ,
      3. Patterson SD
      . Sample sizes for estimation in clinical research. Pharm Stat 2004;3:213–5.doi:10.1002/pst.125
      OpenUrl
    67. 67.↵
      2. Julious SA
      . Sample size of 12 per group rule of thumb for a pilot study. Pharm Stat 2005;4:287–91.doi:10.1002/pst.185
      OpenUrlCrossRefWeb of Science
    68. 68.↵
      2. van Belle G
      . Statistical rules of thumb. Second Edition, 2008:1–272.
    69. 69.↵
      S Isaac WBM. Handbook in research and evaluation: a collection of principles, methods, and strategies useful in the planning, design, and evaluation of studies in education and the behavioral sciences, 1995.
    70. 70.↵
      2. Lancaster GA ,
      3. Dodd S ,
      4. Williamson PR
      . Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract 2004;10:307–12.doi:10.1111/j.2002.384.doc.x
      OpenUrlCrossRefPubMedWeb of Science
    71. 71.↵
      2. Salter K ,
      3. Jutai JW ,
      4. Teasell R , et al
      . Issues for selection of outcome measures in stroke rehabilitation: ICF activity. Disabil Rehabil 2005;27:315–40.doi:10.1080/09638280400008545
      OpenUrlCrossRefPubMedWeb of Science
    72. 72.↵
      2. Cramer GD ,
      3. Cantu JA ,
      4. Pocius JD , et al
      . Reliability of zygapophysial joint space measurements made from magnetic resonance imaging scans of acute low back pain subjects: comparison of 2 statistical methods. J Manipulative Physiol Ther 2010;33:220–5.doi:10.1016/j.jmpt.2010.01.009
      OpenUrlPubMed
    73. 73.↵
      2. Owens EF ,
      3. Hart JF ,
      4. Donofrio JJ , et al
      . Paraspinal skin temperature patterns: an interexaminer and intraexaminer reliability study. J Manipulative Physiol Ther 2004;27:155–9.doi:10.1016/j.jmpt.2003.12.019
      OpenUrlPubMed
    74. 74.↵
      2. Koo TK ,
      3. Cohen JH ,
      4. Zheng Y
      . A mechano-acoustic indentor system for in vivo measurement of nonlinear elastic properties of soft tissue. J Manipulative Physiol Ther 2011;34:584–93.doi:10.1016/j.jmpt.2011.05.009
      OpenUrlPubMed
    75. 75.↵
      2. Clare HA ,
      3. Adams R ,
      4. Maher CG
      . Reliability of detection of lumbar lateral shift. J Manipulative Physiol Ther 2003;26:476–80.doi:10.1016/S0161-4754(03)00104-0
      OpenUrlPubMed
    76. 76.↵
      2. Houweling T ,
      3. Bolton J ,
      4. Newell D
      . Comparison of two methods of collecting healthcare usage data in chiropractic clinics: patient-report versus documentation in patient files. Chiropr Man Therap 2014;22:32.doi:10.1186/s12998-014-0032-9
      OpenUrl
    77. 77.↵
      2. Battaglia PJ ,
      3. Maeda Y ,
      4. Welk A , et al
      . Reliability of the Goutallier classification in quantifying muscle fatty degeneration in the lumbar multifidus using magnetic resonance imaging. J Manipulative Physiol Ther 2014;37:190–7.doi:10.1016/j.jmpt.2013.12.010
      OpenUrlCrossRefPubMed
    78. 78.↵
      2. Russell BS ,
      3. Muhlenkamp KA ,
      4. Hoiriis KT , et al
      . Measurement of lumbar lordosis in static standing posture with and without high-heeled shoes. J Chiropr Med 2012;11:145–53.doi:10.1016/j.jcm.2012.02.002
      OpenUrl
    79. 79.↵
      2. Koo TK ,
      3. Li MY ,
      4. My L
      . A guideline of selecting and reporting intraclass correlation coefficients for reliability research. J Chiropr Med 2016;15:155–63.doi:10.1016/j.jcm.2016.02.012
      OpenUrlCrossRefPubMed
    80. 80.↵
      2. Crosby RD ,
      3. Kolotkin RL ,
      4. Williams GR
      . Defining clinically meaningful change in health-related quality of life. J Clin Epidemiol 2003;56:395–407.doi:10.1016/S0895-4356(03)00044-1
      OpenUrlCrossRefPubMedWeb of Science
    81. 81.↵
      2. Moher D ,
      3. Hopewell S ,
      4. Schulz KF , et al
      . CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c869.doi:10.1136/bmj.c869
      OpenUrlFREE Full Text
    82. 82.
      2. Roalf DR ,
      3. Moberg PJ ,
      4. Xie SX , et al
      . Comparative accuracies of two common screening instruments for classification of Alzheimer’s disease, mild cognitive impairment, and healthy aging. Alzheimers Dement 2013;9:529–37.doi:10.1016/j.jalz.2012.10.001
      OpenUrlCrossRefPubMedWeb of Science

    Footnotes

    • Contributors FB and CS-A conceived the study design. FB wrote all drafts of the protocol with significant contributions from CS-A at all stages. Both authors contributed, read and approved the final manuscript.

    • Funding Commission for Technology and Innovation CTI (Bern, Switzerland)–project no: 18776.1PFLS-LS.

    • Disclaimer CTI has not been involved in the design or undertaking of the study and will not be involved in the analysis or preparation of publications resulting from the research.

    • Competing interests None declared.

    • Patient consent Not required.

    • Ethics approval Ethical approval was obtained from the responsible Swiss ethics committee (EC): EC Nordwestschweiz (2017/200).

    • Provenance and peer review Not commissioned; externally peer reviewed.