Article Text
PDF
Abstract
Introduction Ehlers-Danlos syndromes (EDS)/generalised hypermobility spectrum disorders (G-HSD) affect the connective tissue of the body and present with a heterogeneous set of symptoms that pose a challenge for diagnosis. One of the main diagnostic criteria of EDS/G-HSD is generalised joint hypermobility, which is currently assessed by clinicians during a physical exam. However, the practice for measuring joint hypermobility is inconsistent between clinicians, leading to high inter-rater variability. Often patients are misdiagnosed with EDS/G-HSD based on an incorrect hypermobility assessment, leading to increased referral rates and resource utilisation at specialised EDS clinics that results in unnecessary emotional distress for patients. An objective, validated and scalable method for assessing hypermobility might mitigate these issues and result in improved EDS/G-HSD patient care.
Methods and analysis This study will examine the use of videos obtained using a smartphone camera to assess the range of motion (ROM) and hypermobility of the joints assessed in Beighton score and more (spine, shoulders, elbows, knees, ankles, thumbs and fifth fingers) in individuals with suspected EDS/G-HSD. Short videos of participants will be captured as they undergo a formal assessment of joint hypermobility at the GoodHope EDS Clinic at Toronto General Hospital. Clinicians will measure the ROM at each joint using a clinical-grade goniometer to establish ground truth measurements. Open-source human pose-estimation libraries will be used to extract the locations of key joints from the videos. Deterministic and machine learning systems will be developed and evaluated for estimating the ROM at each joint. Results will be analysed separately for each joint and human pose-estimation library.
Ethics and dissemination This study was approved by the Research Ethics Board of the University Health Network in Toronto on 26 April 2022. Participants will provide written informed consent. Findings from this study will be published in peer-reviewed journals and presented at conferences.
Trial registration number NCT05366114.
- GENERAL MEDICINE (see Internal Medicine)
- Quality in health care
- REHABILITATION MEDICINE
- GENETICS
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
STRENGTHS AND LIMITATIONS OF THIS STUDY
This study will examine the use of standard colour video to quantitatively assess the range of motion in individuals with suspected generalised joint hypermobility.
The movement at seven distinct joints will be analysed (spine, shoulders, elbows, knees, ankles, thumbs and fifth fingers) as per 2017 Ehlers-Danlos syndromes criteria, and where appropriate and consistent with standard clinical practice, compared with gold-standard goniometer measurements taken by clinicians.
Multiple open-source pose-estimation libraries for tracking the positions of the major joints will be systematically investigated in this population.
A limitation of this study is that all data will be collected at a single site.
Background
Ehlers-Danlos syndromes (EDS) are a heterogeneous group of inherited genetic disorders that affects connective tissue throughout the body in various forms.1 One of the most common characteristics of EDS is multiple hypermobile joints that may be unstable and subluxate or dislocate, hyperextensible or fragile skin, organ or systems dysfunction and widespread pain.1 As of 2017, there are 13 recognised subtypes of EDS. While the other 12 subtypes can be confirmed with genetic or molecular analysis, the gene(s) associated with the most common subtype, hypermobile EDS (hEDS), have not yet been identified.1
Currently, hEDS is diagnosed through a clinical interview, medical history and a physical exam as per the 2017 hEDS criteria.1 Hypermobility is a physical manifestation with a wide clinical spectrum encompassing from healthy normal individuals to the ones affected by either inherited or non-inherited medical disorders. Hypermobility is influenced by genetic and somatic factors such as age, gender and ethnicity, and can be a normal familial trait or the result of physical training. Conversely, individuals can present with generalised joint hypermobility and chronic pains although fall short of meeting the formal diagnostic criteria of hEDS. Individuals in this subgroup are currently classified as generalised hypermobility spectrum disorders (G-HSD) and are thought to be on the spectrum of EDS. One of the key clinical tools for assessing generalised joint hypermobility commonly associated with EDS/G-HSD is the Beighton exam.2 This exam assesses the apposition of the thumb to forearm, extension of the fifth finger beyond 90°, extension of knees and elbows beyond 10°, and forward flexion; and assigns a positive score for each joint that meets the criteria for hypermobility. Often, hypermobility of other large and small joints not included in the Beighton score are also commonly assessed in specialised EDS clinics.3
The Beighton score is a validated measure of generalised joint hypermobility,4 however, there continue to be several challenges with its widespread and consistent application by clinicians.5 While the Beighton score is only clinically validated when assessed using a goniometer, the use of goniometer is not standard practice for all clinicians, leading to inaccurate measurements and higher inter-rater variability in Beighton score reporting.6 In the clinical setting, physicians often perform only a single Beighton manoeuvre of thumb to forearm apposition and perform brief visual scan of other joints as a rapid method of generalised hypermobility assessment, introducing a significant diagnostic error and potential misdiagnosis.7
On one side, increased awareness of the multisystem manifestations of EDS/G-HSD through online access has contributed to a significant increase in the frequency of referrals to specialised genetics, rheumatology, pain and EDS services. On the other side, physicians are often not competent in assessment of Beighton score using a goniometer and may either overlook EDS/G-HSD as a potential diagnosis or overscore on Beighton scale, which results in diagnostic error. A suspicion of EDS/G-HSD is commonly confirmed based on hypermobility with use of Beighton score for confirmation of generalised joint hypermobility. This inaccurate or lack of assessment of joint hypermobility either creates undue overcrowding of referrals creating backlogs and/or delay in provision of care to individuals living with EDS/G-HSD.
To address the issue of inconsistent assessment of generalised joint hypermobility in individuals with suspected EDS/G-HSD, this study proposes the development of an objective system for estimating the range of motion and the Beighton score from video data. Cameras are unobtrusive and readily available in many consumer devices such as phones, tablets and webcams, allowing for easy adoption in future iterations of the tool. In this preliminary study, the performance of a video-based system for Beighton score assessment will be compared with expert clinician assessments conducted at a specialised EDS clinic.
To extract meaningful data from recorded videos, open-source human pose-estimation libraries will be used. Pose-estimation libraries operate on an input image or video and output the location of key joints in the image coordinate system. Well-established pose-estimation libraries that provide locations of the major joints of the entire body include OpenPose, Detectron, AlphaPose and MediaPipe Body.8–11 Specialised pose-estimation models have also been developed for tracking hand pose with greater precision.12 13 These human pose-estimation models have been trained and evaluated on a large corpus of ‘in-the-wild’ videos, which include many different individuals performing a variety of activities in uncontrolled environments.14 15 However, a limitation of these open-source pose-estimation libraries is that they have not been explicitly trained on data from populations with individuals presenting with generalised joint hypermobility. Therefore, the tracking ability of pose-estimation libraries in these extreme ranges is not well understood. This work will investigate the performance of several open-source human pose-estimation libraries for tracking joint locations in hyperextended positions and the subsequent range of motion estimated based on the detected positions.
Methods
Study design
This is a cross-sectional, single-centre, observational study that will be used to assess the feasibility of measuring generalised joint hypermobility using video data. Approximately, 225 participants will be recruited from the pool of patients undergoing an initial physical examination at the GoodHope EDS Clinic at Toronto General Hospital.3
Objectives
The primary objective of this study is to validate the use of a vision-based tool for assessing Beighton score in a population of adults with suspected EDS. A video tool-based pose tracking system that estimates whether each of the knees, elbows, thumbs, fifth fingers and spine are hypermobile will be developed. The estimates provided by this system will be compared with the clinicians’ determination of joint hypermobility using the Beighton criteria. Although not part of the Beighton assessment, the ankle and shoulder joints will also be assessed. In accordance with the cut-off thresholds used in the GoodHope EDS Clinic, shoulder flexion greater than 180° and active ankle dorsiflexion greater than 15° from vertical were deemed as positive findings. Goniometric measurements for both these angles will be done in accordance with standard methods.16
To facilitate the development of the vision-based tool, this study will also investigate how accurately the range of motion (in degrees) at key body joints in a population of adults with suspected EDS can be estimated from video. The range of motion estimated by the vision-based system will be compared with ground-truth measurements taken by expert clinicians using a digital goniometer.
The agreement between the system predictions and clinician measurements will also be analysed separately for each joint to identify whether there are systematic differences in accuracy for the joints of interest. As part of this work, several open-source human pose-estimation libraries available for research use will be employed to extract joint trajectory data from the raw video data, and the performance of the system when using each will be evaluated.
Recruitment and participation criteria
Patients scheduled for an in-person physical assessment at the GoodHope EDS Clinic at Toronto General Hospital will be asked if they are interested in learning about this research study by a clinician within their circle of care. Those who indicate interest will be approached by a research assistant who will explain the details of the study and provide individuals with a consent form to review. Participants will be given adequate time to reach an informed decision with respect to study participation. Participants may withdraw consent for study participation at any point and any collected data will be deleted and excluded from all future analyses.
Inclusion criteria
The inclusion criteria for this study are:
Adults over 18 years of age.
Referred to the EDS clinic (referral to clinic requires a Beighton score greater than 3, as assessed by the referring clinician).
Able to provide informed written consent in English.
Exclusion criteria
Participants who do not consent or for whom the clinician deems the manoeuvres to be unsafe will not be included.
Consent process
Patients are first involved in this study when they are approached for potential study participant on visiting the EDS clinic. They are provided information about the study objectives and are given the opportunity to ask any questions related to the study or their participation. As part of the consent process, the participants are asked whether they wish to receive an update about the study results through email or physical mail when they become available.
Patient and public involvement
Patients and/or the public were not involved in the design or conduct of this research.
Sample size planning
As calculated for the Mann-Whitney U test, a non-parametric method for comparing whether two populations come from the same distribution, at alpha of 0.05 and power of 80%, a sample size of 221 facilitates discrimination of distributions with medians 2° apart with an SD of 7°. While it is difficult to approximate the expected SD in the estimates and measurements and how they will vary for each joint assessed, with our selected sample size of 225, the difference between the estimated range of motion at each joint and the corresponding goniometer measurement can be reliably compared for an effect size greater than 0.286.
In a regular week, the GoodHope EDS Clinic assesses 10–12 new patients. Based on a recruitment rate of 7 participants a week, we aim to collect data from a minimum of 225 participants during an 8-month data accrual process.
Data collection
Participants who agree to participate in the study will undergo study-specific procedures in addition to their regular standard of care during their scheduled physical assessment at the EDS clinic. Specifically, video recordings and goniometer measurements of their range of motion will be taken as they undergo the assessment of joint hypermobility.
The participant will be asked to perform each manoeuvre for assessing joint hypermobility twice. In the first repetition, the clinician will measure the range of motion at each joint (except the spine and thumbs, where this is not standard practice) using a goniometer. The clinician will also note whether each Beighton manoeuvre is positive or negative. During the second repetition, the participant will be recorded using the camera while they perform the joint assessment.
A total of 13 individual joints will be assessed: the spine, both ankles, knees, elbows, shoulders, thumbs and fifth fingers. The movements assessed are summarised in table 1.
Joint hypermobility assessments for patients at the GoodHope EDS Clinic at Toronto General Hospital
Clinical data items
During the first repetition of the joint assessment (which will not be video recorded), the clinician will measure the range of motion at the fifth finger, knee, elbow, shoulder and ankle joints in degrees based on standard criteria16 using a JAMAR Plus+clinical-grade digital goniometer (figure 1).
A mobile phone camera mounted on a TRIPOD (top) will be used to record videos of the joint hypermobility assessments. A 12.5” JAMAR Plus+clinical-grade digital goniometer (bottom) will be used to measure the range of motion at each joint. TRIPOD, Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis.
When applicable, the clinician will assess whether the joint fulfils the criteria for hypermobility as established by the Beighton criteria. Finally, age and sex, factors which are known to affect joint laxity, will be recorded for each participant.17 An encoded identifier of the clinician performing each assessment will also be recorded for each participant.
Video recordings
A consumer-grade mobile phone camera (Motorola Moto G Pure or Motorola E5 Play) will be used to record the videos of individuals performing the second repetition of the joint assessment. The phone will not be connected to any wireless internet or cellular networks, and videos will be transferred to a secure location on the hospital network through a USB cable after each day of data collection.
The mobile phone will be mounted on a tripod and will be positioned so it does not contact the participant or impede their range of motion. Dark tape placed on the floor will be used to help standardise the distance between the camera and the participant during data collection. If needed, the camera position will be adjusted slightly to accommodate taller and shorter participants.
The equipment used in this study is shown in figure 1. The target distance between the camera and participant from which the videos will be taken is summarised in table 2.
Target distance between mobile phone and participant during video data collection by joint
A member of the research team will enable video recording during the generalised joint hypermobility examination. Video recording will be stopped and restarted for each joint assessment. The position of the camera will be adjusted as needed between the assessment of each joint. Videos will be recorded at a resolution of 1920×1080 pixels and a frequency of 30 Hz. Whenever possible, the videos will be framed such that only the participant is visible in the video.
Anonymity of participants
Due to the identifying nature of video recordings, the participants’ faces will be blurred and the original, unblurred videos will be deleted when all data analysis is complete. Furthermore, any videos or images used in presentations or publications will have the face of the participant cropped or blurred. Only individuals who provide consent to use their videos in presentations and publications will be included. No information identifying the participant will be transferred beyond the investigators in this study.
Confidentiality of data
Anonymity will be strictly maintained throughout the research process. Data handling and record keeping will follow the protocol of the hospital’s policy and every effort will be made to ensure confidentiality.
All data sources will be deidentified and coded with participants’ unique study number. The key linking the participant (and clinician) ID to participant (and clinician) name and other identifying information will be encrypted and password and stored in a separate location. The study data will be stored in a secure database and kept for 10 years following study completion, until March 2033.
All recorded data, including video files, will be transferred from the recording device to a secure hospital server-based storage location using a USB cable. Access to this server storage will be restricted to study team members and will only be accessible through individual accounts with appropriate credentials and valid authentication. All data analyses will be performed on hospital equipment, ensuring that workstations are encrypted, password protected and restricted to study personnel only.
Data monitoring
No specific data monitoring is required by the REB due to the observational nature of this study.
Data analysis
Pose-estimation from video
In this study, whole-body pose-estimation libraries will be used to extract joint positions in videos of the spine, shoulder, elbow, knee and ankle assessment videos, while specialised hand models will be used for the videos of the thumbs and fifth fingers. Figure 2 shows an example of the output of the MediaPipe Body and MediaPipe Hands pose-estimation libraries on frames from preliminary (non-patient) data collection.
Examples of detected joint positions as output from the MediaPipe Body (left) and MediaPipe Hands (right) human pose-estimation libraries.
Prediction of joint range of motion
After extracting the joint trajectories, the range of motion at each joint will be predicted using deterministic geometric methods, as well as by training appropriate machine learning models using the joint trajectories as input. The performance of each method will be assessed by comparing the predicted range of motion to the angle measured by the clinician using the goniometer at each joint (except the spine and thumbs), as well as the binary determination of whether the joint is hypermobile based on clinical criteria. The predictions will be analysed separately for each joint to understand whether there is a difference in performance across joints.
Effect of pose-estimation library on system performance
As only the joint trajectories extracted from video (not the video itself) will be used by the system to predict the range of motion, it is important to understand if there are significant differences in how well these libraries track movement in hypermobile and non-hypermobile joints. To evaluate the pose-estimation libraries, ground-truth annotations of the joint positions will be manually labelled in a subset of frames of the recorded videos. The difference between the ground-truth annotations and the positions predicted by each pose-estimation library will be calculated. Analogously, the performance of the overall system for range of motion prediction will be evaluated when using joint trajectory input from each pose-estimation library by comparing to the ground-truth data measured with the goniometer.
Fine-tuning a population-specific pose-estimation model
Depending on the performance of the pose-estimation libraries in tracking the joint positions during the assessments of joint hypermobility, it may be necessary to fine-tune these models. By using manually annotated data from individuals with suspected EDS/G-HSD, the pose-estimation libraries can be refined to more accurately estimate the joint positions in videos of hyperextended joints. Previous work on fine-tuning models for facial keypoint prediction found that a sample of 40 patients from the target clinical population is sufficient to fine-tune the model, with no additional significant improvement if more patients are included.18 Therefore, if needed, a subset of 40 participants will be used to finetune a pose-estimation model optimised for use with the dataset collected as part of this study. All analyses will be repeated using the fine-tuned pose-estimation model to evaluate any performance improvement as a result of domain-specific fine-tuning. The evaluation of the fine-tuned model will be done using k-fold cross validation to ensure that the participants on which it is tested were not seen during training.
Statistical analysis
Measures of central tendency (eg, mean, median, mode) and spread (eg, SD, range and quartiles) will be used to summarise the data. The normality of the range of motion at each joint will be evaluated and appropriate parametric or non-parametric tests will be used to assess whether there are significant differences between joint hypermobility in the pathological and non-pathological groups, as measured by the goniometer and as predicted by the machine learning system. The correlation between the measured and estimated range of motion in each joint will also be calculated for each joint.
For Beighton score prediction, we aim to calibrate our vision-based tool to be able to achieve a sensitivity of at least 0.7 while screening out half of the participants as being part of the negative class. Based on historical clinical encounters in our setting, approximately 85% of the patients have a negative Beighton exam on assessment at the clinic, so this target is designed to reduce the number of patients with ultimately negative Beighton exams that are referred to the clinic, while also ensuring that those with a positive exam are not missed.
The performance of the system in estimating the joint range of motion and Beighton scores will also be evaluated separately for each clinician rater. This will aid in identifying if there are systemic differences in the performance of the tool access clinicians.
Ethics and dissemination
This study was approved by the University Health Network Research Ethics Board in Toronto on 26 April 2022 (Study ID - 22-5073.2). Data collection will take place over 1 year and began in June 2022. Results are anticipated in Spring 2023.
Dissemination of results
A preliminary manuscript presenting the performance of several open-source human pose-estimation libraries in predicting joint locations in hypermobile individuals is anticipated on a subset of data collected during this study. A comprehensive manuscript on the development and evaluation of the video-based tool for hypermobility assessment will be prepared after all data collection is complete.
Participants who indicate that they wish to receive a copy of the study results as part of the consent process will receive a summary of the findings at the conclusion of the study.
Discussion
Benefits of the study
This study will be the first to assess the feasibility of measuring joint hyperextension in individuals with suspected generalised joint hypermobility using consumer-grade video. A vision-based system will be developed and evaluated in individuals from the target population while they undergo a formal clinical assessment at a specialised EDS clinic. Measurements of the range of motion at each joint will be measured using a digital goniometer by clinicians with expertise in working with this population, so accurate and consistent ground-truth annotations will be available in this study.
The vision-based system developed as part of this study will be evaluated when using different human pose-estimation libraries and will provide insight into how well each open-source library is able to track movement of individuals with suspected joint hypermobility. Finally, all evaluations will be performed separately for each joint to investigate if there are significant differences based on the joint of interest.
Limitations
A limitation of this study is that all data will be collected at a single site. While the GoodHope EDS Clinic at Toronto General Hospital sees patients from across the province of Ontario, all videos will be taken in the same physical setting. Data recorded within the same environment by the same individuals and with the same equipment removes a layer of variability in the dataset, potentially impacting the generalisability of the developed tool. However, as the raw recorded videos will be processed with robust open-source human pose-estimation libraries which have not been exposed to data from this study before, it is anticipated that joint trajectories extracted from data collected at future sites will be of similar quality.
Furthermore, the GoodHope EDS Clinic only accepts referrals from adult patients. Adolescents may also present with generalised joint hypermobility, but are not eligible to participate in this study. Therefore, future work is needed to understand the performance of the developed system in a younger population.
Conclusion
If the system developed as part of this study achieves the established performance benchmarks, the next phase of this work will focus on developing a mobile phone app or web-based application facilitating the use of the tool to aid in the referral of patients to specialised EDS clinics. When deployed, the application would allow primary care physicians or patients to upload videos and receive a standardised score of joint hypermobility. This would aid in ensuring that only those individuals with higher probability likelihood of being diagnosed with EDS or G-HSD are referred to the specialised clinic. Currently, the wait time for patients to be seen at the GoodHope EDS Clinic at Toronto General Hospital is over 2 years, however, less than 30% of individuals referred to the clinic are ultimately diagnosed with EDS/G-HSD. A standardised vision-based assessment of joint hypermobility may help to reduce wait times for assessment at specialised EDS clinics, improve resource utilisation and expedite timely care for individuals with EDS or G-HSD.
Ethics statements
Patient consent for publication
Acknowledgments
The authors acknowledge the generous donation from the Canrector Foundation for this project.
Footnotes
Twitter @mittalnimish
Contributors NM, AD and BT contributed to the conception and organisation of the study. NM, AS, AD and BT contributed to the design of the study and preparation of study procedures. As physicians at the GoodHope EDS clinic at Toronto General Hospital, NM and HC provide expertise in the operation of the clinic and specialised considerations when working with the population seen at the clinic. NM and AS are responsible for data collection and writing. AS will be responsible for the data processing and statistical analyses. NM, AS, AD, HC and BT will contribute to the interpretation of data and results of the statistical analysis. All authors commented on the study protocol and approved the final version.
Funding This research is funded by a philanthropic donation from Canrector Foundation.
Disclaimer The funding bodies above do not have any role in the design of the study or collection, analysis, and interpretation of data.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Provenance and peer review Not commissioned; externally peer reviewed.