Article Text

Download PDFPDF

Using an interactive virtual environment to integrate a digital Action Research Arm Test, motor imagery and action observation to assess and improve upper limb motor function in patients with neuromuscular impairments: a usability and feasibility study protocol
  1. Frank Behrendt1,2,
  2. Corina Schuster-Amft1,3,4
  1. 1 Research Department, Reha Rheinfelden, Rheinfelden, Switzerland
  2. 2 University Children’s Hospital Basel, Basel, Switzerland
  3. 3 Institute for Rehabilitation and Performance Technology, Bern University of Applied Sciences, Burgdorf, Switzerland
  4. 4 Department of Sport, Exercise and Health, University of Basel, Basel, Switzerland
  1. Correspondence to Dr Frank Behrendt; f.behrendt{at}


Introduction In the recent past, training systems using an interactive virtual environment have been introduced to neurorehabilitation. Such systems can be applied to encourage purposeful limb movements and will increasingly be used at home by the individual patient. Therefore, an integrated valid and reliable assessment tool on the basis of such a system to monitor the recovery process would be an essential asset.

Objectives The aim of the study is to evaluate usability, feasibility and validity of the digital version of the Action Research Arm Test using the Bi-Manu-Trainer system as a platform. Additionally, the feasibility and usability of the implementation of action observation and motor imagery tasks into the Bi-Manu-Trainer software will be evaluated.

Patients and methods This observational study is planned as a single-arm trial for testing the new assessment and the action observation and motor imagery training module. Therefore, 75 patients with Parkinson’s disease, multiple sclerosis, stroke, traumatic brain injury or Guillain-Barré syndrome will be included. 30 out of the 75 patients will additionally take part in a 4-week training on the enhanced Bi-Manu-Trainer system. Primary outcomes will be the score on the System Usability Scale and the correlation between the conventional and digital Action Research Arm Test scores. Secondary outcomes will be hand dexterity, upper limb activities of daily living and quality of life.

Hypothesis We hypothesise that the digital Action Research Arm Test assessment is a valid and essential tool and that it is feasible to incorporate action observation and motor imagery into Bi-Manu-Trainer practice. The results are expected to give recommendations for necessary modifications and might also contribute knowledge concerning the application of action observation and motor imagery tasks using a training system such as the Bi-Manu-Trainer.

Trial registration number NCT03268304; Pre-results.

  • rehabilitation medicine
  • stroke
  • parkinson’s disease
  • multiple sclerosis

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See:

View Full Text

Statistics from


Interactive virtual training environments become increasingly used in neurorehabilitation to encourage purposeful limb movements.1 There are several potential advantages: they can enable patients to observe their own movements in real time in the virtual environment. Thereby, patients are able to virtually modify their movements and even to perform related tasks that might hardly be executable in reality. They also provide an opportunity for intensive and varied practice at reduced intervention costs, customised exercise protocols, the ability to monitor the exercise and they can increase user motivation.2 3 In a Cochrane review by Laver et al on the usage of virtual reality (VR) in the field of stroke rehabilitation, it was stated that VR was not superior to conventional therapy approaches in improving upper limb function.4 Nevertheless, it might be beneficial as an additional measure and possibly more effective compared with the same dose of additional conventional therapy.4 In patients with Parkinson’s disease, encouraging findings on the potential benefits have also been found5–8 as well as improvements of balance ability in patients with multiple sclerosis.9 On the contrary, there is only limited evidence on the positive impact of such training systems on motor and cognitive functionality in the rehabilitation of traumatic brain injury. Nevertheless, this approach seems to have the potential to provide a worthwhile therapeutic option also for those patients.10

The Bi-Manu-Trainer (BMT) (Reha Stim Medtec AG) is an example of a therapeutic VR system for upper limb training in rehabilitation (figure 1). It provides three feedback modalities (acoustic, visual and sensory) to facilitate performance adaption and offers the opportunity to perform unimanual or bimanual movement tasks of the upper limb with different game options on a computer screen. The device was developed specifically for arm, hand and finger movements only. It allows detecting and displaying even small movement changes.11 It has recently been shown that BMT training and conventional training both seem to have a similar effect on hand dexterity in patients with chronic stroke over a 4-week training period including 16 training sessions.12

Figure 1

Virtual reality training system setup (Bi-Manu-Trainer, Reha Stim Medtec AG). The model wears wireless hand gloves with movement sensors attached. The screen displays real-time hand and finger positions.

However, the BMT system is being continuously revised and its newly developed features are the basis for the planned clinical study. The overall aim is therefore to evaluate the integration of two new software modules. One module was planned to provide the possibility of assessing upper limb motor function using the BMT system (project 1) and the other module integrates action observation (AO) and motor imagery (MI) training into the BMT training (project 2). We hypothesise that the digital Action Research Arm Test (d-ARAT) is a valid and useful assessment tool, and that it is furthermore feasible to incorporate AO and MI into the BMT system.

Project 1: integration of the ARAT into the BMT system

A valid assessment of the upper limb functioning, which can be carried out by the patient himself, might possibly be an appreciable improvement of the BMT training system, especially when applied at home. This could provide both more frequent information about the individual progress and also allow for an automatic individual adjustment of the training software, that is, the level of difficulty. The conventional ARAT13 requires a human examiner to transform observations of a patient’s performance into a score in order to set according treatment goals and select appropriate treatment methods. However, this process is quite time consuming. The new assessment module for the BMT on the basis of the conventional ARAT is envisaged to provide the possibility of self-administering the test with or without examiner supervision. The BMT hardware also comprises wearable sensor technology that can be used to measure motor abilities or to monitor rehabilitation outcome for instance by estimating Fugl-Meyer clinical scores based on motion data.14–16 Accordingly, the novel assessment module was developed to be capable of processing the recorded hand and finger movement data and thereby to judge the movement performance. In this regard, the assessment module for the BMT system was designed to incorporate most of the test items of the established ARAT.

Project 2: AO and MI as integral part of the BMT training

It is now accepted in neurophysiology that the observation of actions performed by others can activate some of the neural structures also responsible for the actual execution of the same actions as there is an overlap of the visual and the motor system.17 Interestingly, during both the actual execution and observation, an increase of force in performing the same movement was found in both hands when compared with a control condition.18 In the field of rehabilitation, AO has been shown to facilitate motor learning and the building of a motor memory trace in normal adults as well as in patients with stroke.19 20

Similarly, MI is a dynamic state during which the representation of a specific motor action is internally activated without any motor output.21 MI requires the conscious activation of brain regions that are also involved in movement preparation and execution, accompanied by a voluntary inhibition of the actual movement.22

With respect to MI, efficacy has frequently been proved for patients with poststroke23 24 with positive effects on upper extremity motor recovery, balance and gait in patients with stroke.25–27

As being valuable methods in rehabilitation, both AO and MI applied in combination can be even more effective concerning the cortical activation pattern and the corticospinal excitability, respectively.28 29 Project 2 of the planned clinical trial, therefore, aims at the implementation of AO and MI tasks into the BMT training.

Primary and secondary objectives

Primary objectives

The overall aim of the study is to evaluate the two novel modules within the BMT system applied in clinical practice. Therefore, the study will investigate the usability of both modules (see table 1 for an overview). It is further planned to evaluate the validity of the new assessment module (project 1), that is, currently being developed on the basis of the conventional ARAT.

Table 1

Overview of study objectives

Secondary objectives

The secondary objective of project 1 is the evaluation of reliability and responsiveness of the assessment module, whereas the secondary objective of project 2 is the evaluation of feasibility and applicability of the AO and MI module. The different feasibility parameters are thought to provide details for an ensuing randomised controlled trial such as most importantly information to enable a sample size calculation.

Methods and design

The study is designed as a single-arm trial. All study parts (see figure 2 for an overview) will conform the guidelines of good clinical practice and the Declaration of Helsinki. Data collection will be performed in a rehabilitation clinic in the German-speaking part of Switzerland.

Figure 2

Study overview: T0, pre-training; T1, measurement after eight training sessions; T2, measurement after eight further training sessions; FU, measurement after 2-month follow-up period. Project 1—Integration of the Action Research Arm Test into the BMT system; Project 2—action observation and motor imagery as integral part of the BMT training. BMT, Bi-Manu-Trainer; FU, follow-up.

Patient selection criteria and recruitment

Patients will be eligible for study participation if they fulfil the selection criteria listed in table 2. They equally apply for both projects. The patient recruitment strategy employs different approaches:

  1. Patients will be recruited from the clinics’ inpatient or outpatient departments by physicians, therapists and nurses.

  2. Patients will be recruited from the clinics’ patient database. Datasets will be screened for study selection criteria by the involved study personnel. If patients are eligible, they will be sent a letter describing the study and including patient information. If patients are interested in participating, they can contact the study personnel in the clinic by phone or email.

  3. Patients will be recruited via a study information flyer provided on the clinic’s homepage and through patient self-help groups. If patients are interested in participating, they can contact the study personnel in the responsible clinic by phone or email.

Table 2

Patient selection criteria

Written informed consent will be obtained from all patients after they received written and oral study information. The procedure will be performed by the study personnel before patient inclusion. Patient information and consent forms both in German can be obtained from the first author. Interested patients will have the choice to either take part solely in the evaluation of the new assessment module (project 1) or both modules (projects 1 and 2, see figure 2). Participation in project 2 includes therapy sessions with the new AO-MI module.

Study procedure

At the first contact, the patient will be informed and invited to partake. If willingness is confirmed within 1–2 days, an appointment for eligibility evaluation and a baseline assessment is made. At the first visit, a baseline assessment shall confirm eligibility according to MMSE, Chedoke Arm and Hand Activity Inventory (CAHAI) and Box and Block Test (BBT) scores. All participating patients will receive their standard therapies during the study as before. The chronological order of all assessments and practice sessions including all outcome measures can be found in table 3.

Table 3

Study procedures

The additional practice with the new AO-MI module in project 2 is described in table 4 using the Template for Intervention Description and Replication checklist and guide.30

Table 4

Description of the action observation and motor imagery (AO-MI) practice

Outcome parameters

All data will be collected on a case report form that will be stored in a locked cabinet and will not be accessible for treating therapists. The standardised case report form can also be obtained from the first author (FB).

Any patient who decides to stop participation in the training will still be invited to all further scheduled measurement events so that the recovery process can be further assessed. See table 5 for an overview of all assessments and the related measurement events.

Table 5

Overview of outcome measures

Primary outcomes

Action Research Arm Test

The primary outcome of interest is the correlation between the scores achieved on the conventional ARAT13 and the d-ARAT. The ARAT, first described by Lyle,31 evaluates 19 tests of arm motor function, both distally and proximally. It is an evaluative measure to assess specific changes in upper limb function among individuals who sustained cortical damage resulting in hemiplegia.31 It is a reliable, valid measure of arm motor status and a valuable tool for characterising clinical state and for assessing spontaneous and therapy-induced recovery.13 The ARAT assesses the patient’s ability to handle objects differing in size, weight and shape and is therefore a valuable arm-specific measure of activity limitation.32 Like other motor assessments, it needs an examiner to transfer patient’s movements into a score.13 With the new digital ARAT, the patient’s upper limb movements will automatically be analysed and rated. The correlation values will be used to determine the validity, test–retest reliability and responsiveness of the d-ARAT.

System Usability Scale

The System Usability Scale (SUS) provides a quick and reliable tool for measuring the perceived usability of any system,33 and is a simple, 10-item attitude Likert scale giving a global view of subjective assessments of usability. It consists of a questionnaire with five response options for respondents; from strongly agree to strongly disagree.34 It can be used on small sample sizes with reliable results and can effectively differentiate between usable and unusable systems.33 35 Furthermore, it is quick and easy for study participants to complete and for administrators to score.36 In the described study, the SUS will be deployed to assess the patients’ judgement of the usability of both new BMT modules.

Secondary outcomes

Box and Block Test

Change in hand dexterity is one of the secondary outcomes of interest. Numerous tests for manual dexterity have been developed, for instance, the BBT.37 38 The BBT is a quick, simple and reliable measurement of manual dexterity and its administration procedure is standardised.39 It consists of moving the maximum number of blocks one by one from one compartment of a box to another of equal size within 1 min.39 It is often assessed in rehabilitation since it is an essential feature of upper limp.39 The BBT has been used in patients after stroke, in patients with multiple sclerosis and traumatic brain injury.40 It is a reliable and valid assessment tool, provides normative data for healthy adult individuals and is frequently used in research and rehabilitation of both children and adults.41

Chedoke Arm and Hand Activity Inventory

Upper limb activities of daily living (ADL) function is planned be assessed using the CAHAI V.9. The CAHAI was introduced to include relevant functional tasks and to be sensitive to clinically important changes in upper limb function.42 It is a validated upper limb measure that uses a seven-point quantitative scale in order to assess functional recovery of the arm and hand.43 The CAHAI scores represent the patients relative ability to independently perform stabilisation or manipulation in ADL with the affected upper limb.44 A score of 1 represents total dependence on another person, and a score of 7 indicates patient’s independence without time or safety concerns or necessary splints or devices. A high interrater reliability and convergent and discriminant cross-sectional validity were established for the CAHAI42 45 and it was found to be more sensitive to clinically important change than the ARAT.42

EuroQol 5-Dimension Questionnaire with 5-level Scale

Participants’ quality of life will be assessed using the EuroQol 5-Dimension Questionnaire with 5-level Scale (EQ-5D-5L) questionnaire.46 47 The EQ-5D is an instrument for the evaluation of quality of live.48 It is based on a descriptive system that defines health in terms of five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression and was designed to measure decrements in health.48 The EQ-5D is a standardised instrument for measuring the generic health status and was first introduced in 1990.46 It has been demonstrated that it is reproducible and valid on the evaluation of quality of life in patients with post-stroke.49 The questionnaire is already translated into several languages, is frequently used and has shown internal consistency when applied to a general population and to groups of patients with various diseases.50

Patient Global Impression of Change

The self-report measure Patient Global Impression of Change (PGIC) reflects a patient’s belief about the efficacy of treatment.51 It is widely used and aims at evaluating all aspects of patients’ health and determining if there has been an improvement or not. PGIC ratings are increasingly being used as ‘gold standard’ in order to assess clinically important change in different conditions.52–54 The patient has to select the one response from the response options on a seven-point scale that gives the most accurate description of the state of health which thus reveals the patient’s rating of the overall improvement.51 55 Possible ratings are ‘very much improved’, ‘much improved’, ‘minimally improved’, ‘no change’, ‘minimally worse’, ‘much worse’ or ‘very much worse’.

Determination of sample size

Project 1

In order to determine an adequate sample size for testing the psychometrics, we assessed the available literature on the ARAT. It has been reported to be valid,13 32 56–58 reliable13 31 32 57 and sensitive to change.59–61 The number of included patients for the estimation of the validity coefficients varies between the cited publications from 12 to 59. However, Hobart et al found that validity estimates would generally be stable in samples n≥80, for 75% of scales in samples of 40 subjects and for 50% of scales in samples of 20.62 They also stated that sample sizes of a minimum of 20 for reliability provided highly representative estimates.62 Thus, after consideration of the available literature and project constraints, we decided to include 75 patients in order to check for the validity of the d-ARAT module and the other objectives of project 1 (table 1). Of these 75 patients, 30 are planned to additionally practise on the AO-MI module in project 2.

Project 2

Further, as it is not intended in the course of this study to apply confirmatory statistics for the evaluation of the AO-MI module a sample size calculation by conducting a power analysis is also not envisaged for project 2. Substantial and reliable data on the effect of BMT practice in adults have not yet been published. Therefore, the sample size determination for project 2 had to rely on according available literature on pilot and feasibility study methodology.63–70 We decided on that basis to include 30 patients into project 2 which aims to evaluate the practice on the AO-MI module.

Statistical analysis

Primary study objectives

A core part of the study is to assess whether the d-ARAT is a valid measurement tool and whether it thereby provides the possibility of accurately evaluating upper limb motor function. As mentioned above, the ARAT has repeatedly been validated before by comparing it to different assessments like the Fugl-Meyer Stroke Assessment.13 Thus, we decided to assess the concurrent validity of the new d-ARAT by comparing its score to the score of the conventional ARAT which will also be assessed at the same measurement events. Validity measures will be evaluated using the Pearson’s correlation coefficient (r)13 which will be considered to indicate high correlation between the conventional ARAT and the d-ARAT and thus a good concurrent validity in case it is greater than 0.75.71 With regard to the evaluation of the usability of both modules, the analysis of the SUS data and the categorisation of the results will be performed on the basis of descriptive statistics.

Secondary study objectives

The intraclass correlation coefficient (ICC) is planned to be used to evaluate test–retest reliability of the d-ARAT assessment module. The ICC has been widely used in conservative care medicine to evaluate interrater, test–retest and intrarater reliability.72–78 For test–retest reliability studies, it was proposed to choose a two-way mixed-effects model along with an absolute agreement definition.79 The latter form of ICC will accordingly be used in this study based on the mean of several measurements. Furthermore, two different approaches of responsiveness recommended by Crosby et al will be applied: Criterion-referenced change (or anchor-based methods) and precision-referenced change (or distribution-based methods).80 For the criterion-referenced change, the d-ARAT change scores will be compared with the BBT change scores using Pearson’s correlation coefficient (r). The precision-referenced change will be analysed by calculating the minimal detectable change. The standardised response mean will be calculated as the ratio of observed change and its SD, which serves as an indicator for the magnitude of change.80

The feasibility of implementing the described practice using the AO-MI module into the BMT training and the use of the digital assessment will be reported using descriptive statistics. This will comprise several outcomes.

  1. Recruitment rate: It will be considered good if more than 85% of the patients addressed partake in the trial.

  2. Eligibility criteria: The applied criteria can be seen as adequate when the selected patients are able to partake in the assessments and also in the intervention if applicable. It is necessary to evaluate whether the criteria a too inclusive or restrictive.

  3. Drop-out rate: A number of less than 15% of the patients leaving the trial will be regarded as good.

  4. Adherence: A participation in at least 80% of the practice sessions will be considered good. As soon as a patient misses one-third of it the respective data will be analysed separately.

  5. The treatment effect will be calculated using Cohen’s d which is the standardised mean difference in order to perform a sample size determination for a randomised controlled trial.

Dissemination policy

The study personnel will adhere to an open access policy:

  • The trialists intend to publish the study protocol and the study results in international open access journals to provide easy access to the study documents for all interested readers.

  • The study is registered in an international open access clinical trial database ( Identifier: NCT03268304).

  • As the study progresses, its methods and preliminary results will be presented at national and international congresses and workshops.

  • After study finalisation and data analyses, all study patients will receive a plain language summary of the study results.

Involvement of professional writers is not intended. No restrictions will be placed on the publication of positive or negative results. Though it is actually not a randomised trial, the study results will be reported in accordance with the guidelines set forth in the 2010 Consolidated Standards of Reporting Trials (CONSORT).81

Criteria for halting the trial

At present, the commercial BMT system has been used for more than 5 years with numerous patients (children, adults) in different acute hospitals and rehabilitation clinics. So far, no adverse events have been reported. However, this study will be halted if any of the following criteria are fulfilled:

  • More than three patients report a sudden onset of or increase in shoulder pain during or just after therapy that is highly likely to be attributable to the use of the BMT, and which does not immediately cease after stopping.

  • More than 25% of the patients report severe cybersickness during BMT training which persists after training is halted.

  • Epileptic seizures in at least two patients are induced directly while using BMT.

Patients reporting the criteria mentioned above will be evaluated by the physician on duty and will be assessed and followed up for the originally planned study duration.

Patient involvement

Patients were not involved in the design of this study.


The aim of the described study is to examine two newly developed software modules for the BMT training device which is already being used in clinics and rehabilitation centres. First, the new d-ARAT assessment module developed on the basis of the established ARAT will be evaluated with respect to its validity, usability, reliability and responsiveness (project 1). Second, the other module which is planned to integrate AO and MI into the BMT system will be checked for its usability, applicability and its feasibility (project 2). Both modules are intended to enhance the BMT system in terms of its functionality and usability also with regard to an individual home use. The individual use could be a small part of the answer of how to deal with the possibly growing number of patients in the future who may need neurorehabilitation training especially when individual care cannot be provided in sufficient quantity and quality. Changing living and working conditions and an improved health-related behaviour are as well leading to an increase in the portion of the population accounted for by the elderly. Therefore, the number of people with disabilities or chronic diseases is constantly rising and the consequences of an altering age structure need to be coped with in particular in the health sector. The rehabilitation technology industry is rapidly evolving and the training systems are constantly enhanced to gain an increased efficacy and usability, and to get well applicable also in home use. The currently planned developments for the BMT system are in line with that process. The results are intended to be used for the evaluation and for supporting the further development and optimisation process of the software and therefore for the patients’ use. The trial will also presumably contribute knowledge concerning the application of relatively new rehabilitation methods such as AO and MI tasks within rehabilitation training using a virtual environment. It is further intended to provide hints on the feasibility and the limitations of a digital motor assessment module. It can be assumed that such a system entails advantages and disadvantages. Obviously, the software algorithm for judging the different movement tasks will be completely objective which prevents a variability of the achieved scores due to interindividual and intraindividual differences of the judging person. On the other hand, the level of precision of the sensors might have an influence on the validity of the new module. However, the successful completion of the planned software development could possibly enrich the range of available assessment and rehabilitation options.


All dissemination will be undertaken using the CONSORT Statement recommendations. Results will be published in peer-reviewed journals and at conference presentations.


  1. 1.
  2. 2.
  3. 3.
  4. 4.
  5. 5.
  6. 6.
  7. 7.
  8. 8.
  9. 9.
  10. 10.
  11. 11.
  12. 12.
  13. 13.
  14. 14.
  15. 15.
  16. 16.
  17. 17.
  18. 18.
  19. 19.
  20. 20.
  21. 21.
  22. 22.
  23. 23.
  24. 24.
  25. 25.
  26. 26.
  27. 27.
  28. 28.
  29. 29.
  30. 30.
  31. 31.
  32. 32.
  33. 33.
  34. 34.
  35. 35.
  36. 36.
  37. 37.
  38. 38.
  39. 39.
  40. 40.
  41. 41.
  42. 42.
  43. 43.
  44. 44.
  45. 45.
  46. 46.
  47. 47.
  48. 48.
  49. 49.
  50. 50.
  51. 51.
  52. 52.
  53. 53.
  54. 54.
  55. 55.
  56. 56.
  57. 57.
  58. 58.
  59. 59.
  60. 60.
  61. 61.
  62. 62.
  63. 63.
  64. 64.
  65. 65.
  66. 66.
  67. 67.
  68. 68.
  69. 69.
  70. 70.
  71. 71.
  72. 72.
  73. 73.
  74. 74.
  75. 75.
  76. 76.
  77. 77.
  78. 78.
  79. 79.
  80. 80.
  81. 81.
  82. 82.
View Abstract


  • Contributors FB and CS-A conceived the study design. FB wrote all drafts of the protocol with significant contributions from CS-A at all stages. Both authors contributed, read and approved the final manuscript.

  • Funding Commission for Technology and Innovation CTI (Bern, Switzerland)–project no: 18776.1PFLS-LS.

  • Disclaimer CTI has not been involved in the design or undertaking of the study and will not be involved in the analysis or preparation of publications resulting from the research.

  • Competing interests None declared.

  • Patient consent Not required.

  • Ethics approval Ethical approval was obtained from the responsible Swiss ethics committee (EC): EC Nordwestschweiz (2017/200).

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.