Article Text
PDF
Abstract
Background The clinical course of acute low back pain (LBP) is generally favourable; however, there is significant variability in the prognosis of these patients. A clinical prediction model to predict the likelihood of pain recovery at three time points for patients with acute LBP has recently been developed. The aim of this study is to conduct a broad validation test of this clinical prediction model, by testing its performance in a new sample of patients and a different setting.
Methods The validation study with a prospective cohort design will recruit 420 patients with recent onset non-specific acute LBP, with moderate pain intensity, seeking care in the emergency departments of hospitals in São Paulo, Brazil. The primary outcome measure will be days to recovery from pain. The predicted probability of pain recovery for each individual will be computed based on predictions of the development model and this will be used to test the performance (calibration and discrimination) in the validation dataset.
Discussion The findings of this study will better inform about the performance of the clinical prediction model, helping both clinicians and patients. If the model’s performance is acceptable, then future research should evaluate the impact of the prediction model, assessing whether it produces a change in clinicians’ behaviour and/or an improvement in patient outcomes.
Ethics and dissemination Ethics were granted by the Research Ethics Committee of the Universidade Cidade de São Paulo, #20310419.4.0000.0064. Study findings will be disseminated widely through peer-reviewed publications and conference presentations.
- back pain
- adult orthopaedics
- musculoskeletal disorders
- spine
- epidemiology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
This study describes the protocol of a broad and detailed validation of a clinical prediction model.
This is the first validation study assessing the performance of the original prediction model designed to predict the likelihood of pain recovery.
The sample will come from a different country and a different setting, providing a robust test of the broad external validity of the prediction model.
The participants of the present study will not come from a randomised controlled trial, differing to the development study.
Definitions of recovery may vary within literature.
Background
The clinical course of acute low back pain (LBP) is generally favourable1 2; however, there is significant variability in the prognosis of these patients. Approximately 80% of patients with acute LBP recover within 6–12 weeks.1 However, even during this period, individuals may present different pain trajectories: some recover within a few days, others recover more slowly and a third group does not recover.3 The ability to identify the likelihood of each patient recovering at specific time points would be valuable to patients and clinicians to help in decisions about the amount and type of treatment to provide.4 5 For example, a patient with a favourable prognosis and high predicted probability of recovery may receive simple baseline care rather than additional interventions. In contrast, a patient with low probability of recovery may be more likely to receive additional interventions despite the costs and time involved.4
Clinical prediction models help to inform possible estimates of an outcome at a given time point.5 A good prediction model needs to be easy to use, discriminate well between patients at different risk levels and provide accurate predictions of the outcome.6 7 Validation studies evaluate the performance of the original model using data from a different sample of patients to ensure that similar results are replicated in a different patient sample or in a different setting.6 The impact study is the final step to identify whether the clinical prediction model produces a change in the clinician’s behaviour or an improvement in patient outcomes.8 After this step, the prediction model can be recommended for use in clinical practice.8
There are existing clinical prediction models reported in the literature to inform about prognosis of patients with LBP.9–13 A recent systematic review14 identified seven clinical prediction models developed to inform about prognosis of patients with LBP, however, they have important limitations. Most of them do not achieve acceptable performance.14 All models focused on predicting poor (eg, persistent or non-recovering pain and disability) and long-term outcomes (eg, 6 and 12 months).14 Although long-term prognosis is helpful, this information may not be the most important information to patients with an acute episode of LBP, such as those presenting to emergency or primary settings.14
A new clinical prediction model has been recently developed to predict the likelihood of pain recovery at three time points for patients with acute LBP.4 This prediction model was designed to be used in a clinical review, which was performed 1 week after the first visit. The prediction model indicates the likelihood of pain recovery at 1 week, 1 month and 3 months after the clinical review visit.4 The reason for developing the model was primarily so that pain intensity change during the first week could be used as a predictive variable. Pain intensity change was previously described as an important predictor of outcomes for patients with LBP.15 16 The second reason for the development of this prediction model is that it aligns with the recommendations from guidelines for the management of acute LBP, which suggest minimal intervention followed by review at 1–2 weeks.2 17
A recent study13 performed narrow validation testing of an updated version of this clinical prediction model. The study used previously collected data from patients presenting to primary care practices similar to the development study, but from a different country.13 Three of these variables were categorised differently in the Danish validation dataset (duration of current episode, number of previous episodes and depression). It was necessary to slightly modify the prediction rule using the new sample before testing.13 A new broad validation study with variables coded exactly as per the original prediction model, and including patients from both a different setting and a different country, would provide stronger evidence on the generalisability of the original prediction model.
Therefore, the aim of this study is to conduct broad validation test of this clinical prediction model, which aims to predict pain recovery in patients with acute LBP, by testing its performance in a new sample of patients from emergency care departments in Brazil.
Methods
Development study methods
The data for the development study came from a randomised controlled trial of paracetamol compared with placebo for the management of acute LBP.18 In the development study, participants were recruited by general practitioners, physiotherapists and pharmacists in the greater metropolitan region of Sydney, Australia, between 2009 and 2012.18 The development study included patients with recent onset non-specific acute LBP, with or without leg pain, less than 6 weeks’ duration, and with moderate pain intensity (measured by an adaptation of item 7 of the SF-36)19 from primary care practices in the greater metropolitan region of Sydney, Australia.19 Only participants who had a pain score of ≥2/10 at the first weekly follow-up after initially seeking care were included in the development study (n=1070). Exclusion criteria were suspected serious spinal pathology (eg, metastatic, infection, fracture), current use of an analgesic, spinal surgery in the last 6 months, contraindication to use of paracetamol, use of psychotropic drugs, and current or planned pregnancy. Five predictor variables (duration of current episode, number of previous episodes, depression, pain intensity and pain intensity change over the first week) were included in the final model.
Current validation study methods
Design and ethics
The present validation study will be conducted with a prospective longitudinal cohort design. Ethics were granted by the Research Ethics Committee of the Universidade Cidade de São Paulo, #20310419.4.0000.0064. The approval of the ethics committee can be found in the online supplemental file 1.
Supplemental material
Patient and public involvement
There was no patient involved in the development of the research question.
Source of data/setting
We will recruit patients with recent onset of non-specific acute LBP seeking care in emergency departments from private and public hospitals in Sao Paulo, Brazil.
Participants
In the validation study, the inclusion criteria will be: patients aged between 18 and 80 years, with recent onset non-specific acute LBP with or without leg pain, lasting less than 6 weeks duration, with moderate intensity20 (adapted version of item 7 of the SF-36 Questionnaire),19 as per development study,4 who are seeking care in emergency departments. Only participants who have a pain score of ≥2/10 at the first weekly follow-up after initially seeking care will be included, as per development study. This decision was made as patients with pain scores less than 2/10 were considered unlikely to present for review and if they did, they were unlikely to need further intervention. A clinical prediction model designed for use at 1-week review does not need to include patients who have recovered rapidly and as such may be more discriminative for those with ongoing pain. Non-specific LBP will be defined as pain in the area between the 12th rib and buttock crease, with or without pain in the lower limbs, not attributed to a specific diagnosis (eg, ankylosing spondylitis, vertebral fracture).2 20 21 Individuals will be excluded if they meet any of the following criteria: suspected serious spinal pathology (eg, metastatic, infection, fracture), current or planned pregnancy, non-Portuguese speakers and presence of any renal impairment that could be mistaken as LBP.
Study procedures
The evaluation of possible participants will be based on clinical investigation through a predefined checklist with the inclusion and exclusion criteria. The evaluation will be carried out by physiotherapists with experience in evaluating and researching patients with LBP, in the emergency department, personally and individually. Patients will be informed about the study and invited to participate. The researchers will assess potential participants’ eligibility and ask them to sign the consent form. After the baseline assessment, an assessor will contact the participants weekly by email, text message, WhatsApp or phone call (based on the participant’s preference) to collect data pertaining to the participant’s pain scores until the patient’s recovery or up to 12 weeks. Figure 1 describes the data collection process.
Flow of study procedures. LBP, low back pain.
Outcome measure
The outcome of interest will be ‘days to recovery from pain’. Recovery will be defined as a score of 0 or 1 on a 0–10 Numerical Pain Rating Scale for 7 consecutive days, as per development study. Participants will respond to weekly follow-ups about their LBP intensity (0–10 Numerical Pain Rating Scale) over the past week. Therefore, to be considered recovered for the current study, participants will need to report pain no greater than 1/10 during the past week. In this situation, the patient will also be asked about the date of recovery. Then, a further follow-up will be performed the following week, to confirm the recovery date (recovery for 7 consecutive days). The maximum follow-up time for patients who have not reached recovery will be 12 weeks.
Predictor variables
Five predictor variables (duration of current episode, number of previous episodes, depression, pain intensity and pain intensity change over the first week) will be included in the final model according to the development study.4 The variables of duration of current episode, number of previous episodes and depression will be collected at baseline. The variable pain intensity will be collected at the first week follow-up. The variable pain intensity change over the first week will be calculated by subtracting pain intensity at the baseline from pain intensity at first follow-up. Table 1 describes the predictor variables, how they will be measured and how they will be coded in the analysis.
Initial measurement of predictor variables
Sample size
Previous studies have suggested that sample size requirements for validation prediction models’ studies would require at least 250 events.5 22 Considering previous studies that have used the same outcome measure of the present study,18 23 it is expected that about 80% of patients seeking care for acute LBP will recover from pain at 3 months. In addition, considering the criteria of including only participants who had a pain score of ≥2/10 at the first weekly follow-up, we expect that around 20% of the sample will recover during the first week, and would be excluded of the analysis based on our inclusion criteria. Therefore, from a sample of 420 participants, 336 would be expected to recover at 3 months. In addition, 84 participants (20% of the total sample) would be expected to recover during the first week and would not be part of the analysis. Then, considering the 336 participants minus the 84 that would be excluded, approximately 252 participants would be expected to recover from pain (the event of interest) and included in the analysis, fulfilling the described recommendations.
Data analysis
The probability of pain recovery is a function of the weighted sum of the predictor variables, with each variable weighted by its estimated coefficient, and the baseline survivor function within each stratum of pain intensity. The predicted probability of pain recovery for each individual in the validation dataset will be computed based on predictions of the development model. The resulting predicted probabilities will be used to assess the performance (in terms of discrimination and calibration) of the prediction model by comparison with the observed recovery.
Discrimination indicates how well the model differentiates between those who recover and those who do not.5 Discrimination will be assessed using an overall C-statistic.24 Calibration indicates whether the observed frequencies agree with the predicted probabilities.5 Calibration will be assessed graphically by plotting the observed proportion of patients who recovered against the mean predicted proportion of recovery within deciles of the predicted probabilities. This will be done for the development sample and also for the validation study sample at three time points (1 week, 1 month and 3 months). The observed proportion of participants recovered in each decile will be computed using the Kaplan-Meier method. All analyses will be performed using IBM SPSS software V.20.0.
Discussion
This study protocol provides a detailed description for validation of an existing clinical prediction model4 to predict pain recovery in patients with acute LBP. This study will be conducted in a different sample of patients, from emergency departments of public and private hospitals in an emerging country, and the findings of the study will better inform about the performance of the existing clinical prediction model.
This is the first validation study assessing the performance of the original prediction model designed to predict the likelihood of pain recovery, at key time points, in patients with acute LBP. It is one of few validation studies of prediction models designed to inform about prognosis of patients with LBP,25 and the only validation study of a prediction model designed to predict recovery in people with acute LBP.
This prospective study is designed exclusively for the purpose of validation of the prediction model, with a well-defined and representative sample of patients seeking care for their current episode of LBP. The sample will come from a different country with a different setting, providing a robust test of the broad external validity of the prediction model. All measures of outcome and predictor variables will be collected exactly the same way as per development study.
Our study also has some potential limitations. First, the participants on whom the original prediction rule was developed came from a randomised controlled trial investigating paracetamol compared with placebo. Participants enrolled in clinical trials may differ from patients not included in trials, and the interventions provided to both groups will be different. It is possible the probability of recovery in this cohort may differ from patients not included in a trial. However, the characteristics of the 1650 participants in the randomised controlled trial which the data of the development study came from and an inception cohort of 969 consecutive participants of acute LBP in the prognosis study of Henschke et al26 they appear similar on most demographic and clinical characteristics. Second, there is no widely accepted definition of recovery; however, the definition used in the present study has been used in previous studies.10 18 23 27
The validation of this prediction model in a different setting and population (emergency department and middle-income country) will have clinical and research implications. Considering a real clinical scenario at an emergency department where patients are not likely to follow-up with their emergency department provider, there is no need of the patient to come back for a second visit. If the patient has a duration of current episode of at least 2 weeks, the clinician can ask the patient about the pain intensity at the moment and the pain intensity considering the previous week. Then it is possible to calculate the pain intensity change from 1 week ago (instead of the first visit) and the time of the visit (instead of 1-week review). This would allow the clinician of the emergency department to use the clinical prediction model in one single visit. While this prognostic information could be used to inform patient and clinician shared decision-making, it is important to note that the current study will not indicate that the use of this model improves clinical decision-making or improves health outcomes, which can only be tested by a model impact study. Future research should test whether the clinical prediction model produces a change in clinicians’ behaviour and/or an improvement in patient outcomes.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Twitter @lcos3060
Contributors TMS, MJH, GAP and LOPC contributed to protocol development and study design. FGS and TMS drafted the manuscript. FGS, TMS, MJH, GAP, LdCMC and LOPC reviewed and revised the manuscript. All authors read and approved the final manuscript and met the ICMJE criteria for authorship.
Funding FGS receives a scholarship from Coordination for the Improvement of Higher Education Personnel (CAPES), Ministry of Education of Brazil. TMdS receives a fellowship from FAPESP (São Paulo Research Foundation) with grant number: 2018/20035–7. The study received a grant from São Paulo Research Foundation (FAPESP) #2019/18187–6.
Disclaimer The funding agencies had no role in the design of the study, collection, analysis, interpretation of data, writing the manuscript and in decision to publish.
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.
Supplemental material This content has been supplied by the author(s). It has not been vetted by BMJ Publishing Group Limited (BMJ) and may not have been peer-reviewed. Any opinions or recommendations discussed are solely those of the author(s) and are not endorsed by BMJ. BMJ disclaims all liability and responsibility arising from any reliance placed on the content. Where the content includes any translated material, BMJ does not warrant the accuracy and reliability of the translations (including but not limited to local regulations, clinical guidelines, terminology, drug names and drug dosages), and is not responsible for any error and/or omissions arising from translation and adaptation or otherwise.