Article Text
PDF
Abstract
Objective High-performance marine craft personnel (HPMCP) are regularly exposed to vibration and repeated shock (VRS) levels exceeding maximum limitations stated by international legislation. Whereas such exposure reportedly is detrimental to health and performance, the epidemiological data necessary to link these adverse effects causally to VRS are not available in the scientific literature, and no suitable tools for acquiring such data exist. This study therefore constructed a questionnaire for longitudinal investigations in HPMCP.
Methods A consensus panel defined content domains, identified relevant items and outlined a questionnaire. The relevance and simplicity of the questionnaire’s content were then systematically assessed by expert raters in three consecutive stages, each followed by revisions. An item-level content validity index (I-CVI) was computed as the proportion of experts rating an item as relevant and simple, and a scale-level content validity index (S-CVI/Ave) as the average I-CVI across items. The thresholds for acceptable content validity were 0.78 and 0.90, respectively. Finally, a dynamic web version of the questionnaire was constructed and pilot tested over a 1-month period during a marine exercise in a study population sample of eight subjects, while accelerometers simultaneously quantified VRS exposure.
Results Content domains were defined as work exposure, musculoskeletal pain and human performance, and items were selected to reflect these constructs. Ratings from nine experts yielded S-CVI/Ave of 0.97 and 1.00 for relevance and simplicity, respectively, and the pilot test suggested that responses were sensitive to change in acceleration and that the questionnaire, following some adjustments, was feasible for its intended purpose.
Conclusions A dynamic web-based questionnaire for longitudinal survey of key variables in HPMCP was constructed. Expert ratings supported that the questionnaire content is relevant, simple and sufficiently comprehensive, and the pilot test suggested that the questionnaire is feasible for longitudinal measurements in the study population.
- content validity
- epidemiology
- fatigue
- high-speed craft
- whole-body vibration
This is an Open Access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/
Statistics from Altmetric.com
Strengths and limitations of this study
The questionnaire was rigorously constructed with its content assessed by field experts and its feasibility pilot tested in a study population sample.
Questionnaire item responses were linked to comeasured craft acceleration and the results showed sensitivity to acceleration exposure.
When combined with objective exposure data, this questionnaire enables quantification of the risk of musculoskeletal pain and impaired performance related to exposure to vibration and repeated shock.
The questionnaire’s content validity is limited by the proficiency of the authors and the expert raters, and the pilot test results by the small sample size.
Introduction
High-performance marine craft personnel (HPMCP) such as coastguards, navy or maritime pilots reportedly suffer from impaired health and performance related to their work at sea. Studies suggest that most of them have had musculoskeletal pain the preceding year,1 work-related injuries which required medical care during their careers,2 and that work-related fatigue commonly degraded their work ability.3–5 Meanwhile, the risks related to the work environment at sea have been poorly investigated and could result from numerous interactive factors. One consistent element claimed to increase these risks is the exposure to vibration and repeated shocks (VRS). Although little is known regarding how far specific VRS components contribute to negative effects, prolonged exposure to whole-body vibration has been linked to musculoskeletal pain and impaired performance in other occupations.6–11 This has resulted in the incorporation of recommendations for maximum daily occupational vibration exposure into international standards and legislation.12–14
Marine personnel are excluded from these statutory exposure limits, however, as compliance with them is infeasible given the available technology combined with the inherent demands of their occupation.14 Those most concerned are likely HPMCP, as they regularly exceed the limits during typical working conditions, even when accounting for shock-mitigation systems.15 16 They also experience some of the highest levels of vibration when compared with that of land-borne personnel with an elevated vibration-related risk for pain.6 17 HPMCP may therefore risk musculoskeletal pain and impaired performance, especially considering their exposure to repeated shock in addition to whole-body vibration. However, the epidemiological data necessary to link causally the contribution of VRS exposure to adverse effects are absent in the scientific literature, and no suitable tools for acquiring such data exist.
Our group recently developed a comprehensive questionnaire that samples information on marine personnel and their working environment, and enables the prevalence of adverse health and performance effects and their association with work exposure to be quantified.18
However, to isolate the causal effects of VRS exposure on health and performance, a complementary, more succinct, instrument with higher resolution is required. Several environmental factors other than VRS likely contribute to adverse effects in marine personnel and needing to be partialled out.19–21 In addition, it is important to select appropriate sampling periods, as sea conditions vary greatly and recall bias decreases measured variable precision.22–24 Also, the longitudinal design necessary for such investigations is prone to data attrition,25 necessitating feasible data collection tools. This study therefore constructed a web-based questionnaire tailored for longitudinal investigation of work exposure, health and performance in HPMCP.
Methods
Design
In three steps, a web-based questionnaire in English was developed, validated and pilot tested in collaboration between the Royal Institute of Technology, Karolinska Institutet, the Swedish Coast Guard and the Norwegian Special Operations Command. Content domains were defined, items were generated and the questionnaire was outlined by a consensus panel. The questionnaire draft was then assessed by experts in an iterative validation procedure, and the validated questionnaire pilot tested in a study population sample.
Consensus panel and expert raters
The present authors constituted the consensus panel: two engineers with theoretical and empirical experience in naval architecture, specialists in high-speed marine craft; and two physiotherapists with experience in epidemiological investigations, biomechanical studies and questionnaire development.
In accordance with previous recommendations based on their knowledge of the content domains, research methodology and statistical analysis,26 10 independent experts from Sweden, Norway and England enrolled for participation: four women and six men (table 1).
Expert characteristics
Development procedure
The questionnaire content was concentrated on key aspects in the previously identified domains of work exposure, health and performance18 to provide a more comprehensive coverage of these features. The literature was reviewed to isolate suitable parameters for domain quantification, and items were selected to reflect central features of the measured constructs while balancing content across domains. Items were evaluated based on their analytical value and the questionnaire was designed to be linked to accelerometer data for objective VRS quantification. Sampling periods were selected to capture accurately the measured variables and to reduce recall bias. To optimise the questionnaire for longitudinal measurements, the balance between data quality and respondent burden was carefully considered, with items selected and web mechanisms implemented to minimise the total number of items. In addition, with the propensity of longitudinal designs for data attrition, optional items were added to facilitate missingness assumptions necessary for result inferences.25 Finally, to evaluate the experts’ concentration level, a control item inquiring about music preference at sea was included in the first questionnaire draft.
Validation procedure
In three consecutive stages, experts assessed individual items by rating their relevance and simplicity on two separate 4-point Likert-type scales: ‘not relevant/not simple’, ‘somewhat relevant/somewhat simple’, ‘quite relevant/quite simple’ and ‘very relevant/very simple’. Ratings were dichotomised so that the two lowest and the two highest options represented non-relevant/non-simple and relevant/simple, respectively.27 28 In addition, experts could comment on individual items and the questionnaire as a whole, and were invited to provide general feedback on the questionnaire’s comprehensiveness and length. Taking into consideration the experts’ feedback, items were revised, added or discarded by the consensus panel between each validation stage. Prior to the third stage, the questionnaire was professionally proofread and implemented online, and the experts were given access to the online version for evaluation in its intended environment.
An item-level content validity index (I-CVI) was computed for relevance and simplicity as the proportion of experts rating an item as relevant or simple, respectively,27 28 with 0.78 selected as the threshold for an acceptable I-CVI.28 29 A scale-level content validity index was calculated as the average across items’ I-CVI (S-CVI/Ave) and as the proportion of items which all experts rated as relevant or simple (S-CVI/UA, scale-level content validity index universal agreement), with selected thresholds of 0.90 and 0.80 for an acceptable S-CVI/Ave and S-CVI/UA, respectively.27 28 A more detailed description of the validation procedure is provided elsewhere.18
Pilot test
To assess the questionnaire’s feasibility and to preliminarily evaluate item properties, it was pilot tested in a convenience sample of eight Norwegian Special Operations Command officers during a marine exercise where high-speed planing craft were regularly operated. Everyone invited agreed to participate in the study. The participants were men aged 28–40 years, with 1–20 years of work experience at sea, who regularly manoeuvred and navigated marine craft.
The questionnaire was completed on the respondents’ personal cellphones, and participants were instructed to complete one section on exposure and performance at the end of each work shift and one section on health once weekly over a 1-month period. In addition, their craft were instrumented to collect the acceleration time-history data at sea to enable data comparison. Following the pilot test period, the subjects provided verbal feedback on the questionnaire.
Results
An overview of the questionnaire construction process is given in figure 1 and the final questionnaire in the online supplementary material 1 and 2.
Supplementary file 1
Flow chart of the questionnaire construction process.
Development
The work exposure domain focused on the crew’s operational environment and contained items related to work: duration, environment and task. One item identified craft ID to permit linkage between questionnaire data and objective data, and a ride-quality item was included as a measure of ride roughness,30 useful both as an indicator of VRS exposure when objective data are unavailable and for identifying acceleration features affecting the perception of ride roughness. Items regarding body posture and crew gear, environmental conditions, mission and work task were included for their biomechanical relevance,5 reported influence on impaired health and performance19–21 and relevance to mental and physical demands, respectively.
The health domain focused on work-related musculoskeletal pain, it being previously associated with VRS exposure and one of the main areas of concern among HPMCP.2 Pain occurrence was considered the main variable and auxiliary items were included to describe its characteristics. In line with established recommendations for chronic pain measurement selected auxiliary items inquired about pain location, pain intensity, pain frequency and physical functioning impairment.31 32 Pain location was mapped with a previously developed 16-zone figure to maintain compatibility with the former questionnaire18 and additional subitems related to the specific locations. Pain intensity was assessed with a standard formulation used to reflect the average pain magnitude over the past week and measured on an 11-point numeric rating scale.31 Pain frequency was quantified by providing a daily schedule split between day and night, allowing for a rapid selection of pain occurrence, and simultaneously permitting quantification of pain patterns and association of pain and exposure. Physical function impairments were considered in relation to reduction in work ability, since this parameter involves both practical and financial ramifications. Finally, one item inquiring about perceived cause of pain was included for its descriptive value.
Performance was mainly measured indirectly via fatigue symptoms, as they have been associated with impaired performance.3 4 33 34 Fatigue is a subjective experience constituting of several dimensions.34 35 Mental fatigue was targeted since it closely reflects performance impairments in common work tasks among HPMCP. A composite summary score derived from four to five items encompassing different aspects of fatigue was considered the most suitable method to capture the latent fatigue construct.34 35 Selected fatigue items were inspired by previous questionnaires,34 36 and adapted to the study population. In addition to the fatigue summary score items, two items for self-rated human and craft performance were included.
Sampling periods were selected considering the characteristics of the measured attributes. Items related to work exposure and fatigue targeted the previous work shift, as work exposure can vary greatly between days, acute fatigue presumably is reversed with rest and both are somewhat diffuse and mundane, which could impede accurate recollection.23 In contrast, musculoskeletal pain items targeted the previous week, as prolonged VRS exposure conceivably causes overload injuries which persist between days, and as a pain event likely is perceived as more distinct and salient, which facilitates accurate recollection.23
To reduce bias related to missing data, one optional item was added with response options defined to support different missing data assumptions.25 Refusal to respond to an item was managed by incorporating a hidden response option (ie, ‘I do not want to answer this question’), which appeared only when respondents attempted to skip an item. Selection of this option strongly suggests that missingness is related to the item itself.
To maintain the respondent burden at an acceptable level, the option to deactivate redundant items (eg, the duration-at-sea item when time at sea is registered elsewhere), a dynamic mechanism which automatically skips redundant items, and only closed-ended response options (ie, predetermined responses selected from a list) were incorporated. With all items active, the dynamic mechanism reduced daily items related to work exposure and performance from 19 to 7 when respondents had not worked at sea, and limited the maximum number of weekly items related to pain to 14 by leading to auxiliary pain items inquiring about the worst and the least painful areas when more than three pain locations were selected.
Validation
The first questionnaire draft contained 28 items (excluding the control item which all experts rated as non-relevant), of which 13 were related to work exposure, 6 to pain, 7 to performance and 2 to missing data. Ratings by 10 experts revealed acceptable I-CVI for simplicity and relevance of 26 items, thereby exceeding the threshold of 0.90 for an acceptable S-CVI/Ave in the first stage. However, 90 item-specific expert comments at this stage prompted further item refinement. Based on this feedback, 18 items were revised, 2 were added to enhance the fatigue summary score and 1 on mission status was discarded as inapplicable to subgroups of the study population.
The second questionnaire draft of 29 items was rated by nine experts, as one expert discontinued the process. Whereas 28 items met the cut-off for an acceptable I-CVI, 45 expert comments again indicated opportunities for further improvements. Accordingly, 12 items were modified and 3 were removed: 1 related to shock mitigation at sea since it was considered redundant, and 2 related to the fatigue summary score since they were found confusing or redundant.
The third and final 26-item questionnaire draft was also rated by nine experts, with 25 items having an acceptable I-CVI for both relevance and simplicity, amounting to an S-CVI/Ave of 0.97 and 1.00, and an S-CVI/UA of 0.85 and 0.96 for relevance and simplicity, respectively. Eight of nine experts commented on the overall questionnaire. All responded that the questionnaire was good to very good; four replied that no additional items were needed while three suggested adding items related to sleep quality, suspension system and in-land work; four suggested that it was of good length while four felt it was slightly too long. The ‘headache’ item (item 12) failed to meet acceptable I-CVI for relevance, was rejected by three of nine experts, but was nonetheless retained for further assessment because of its potential value as a fatigue indicator. Table 2 details the results of the validation process.
Expert ratings across the three validation stages.
Pilot test
The pilot test suggested that the completion time for both questionnaire parts combined was approximately 10 min. Of eight subjects, seven participated in the daily part about work exposure and performance and five in the weekly part about musculoskeletal pain. Over the 1-month period, these respondents completed each part 2–15 and 1–5 times, amounting to a total of 58 and 12 observations, respectively. During the same period, acceleration was registered on 11 occasions between three subjects.
Data obtained indicated that the questionnaire’s psychometric properties were acceptable. Responses had either uniform or unimodal distributions across item categories. The ‘Other’ option available for some items was never selected, and no participants elected to avoid any item response. Exposure-related items registered similar ratings for subjects on the same craft, and there were no contradictory ratings. Of 14 occasions, 7–10 ratings each for ride quality, sea conditions, wind conditions, noise level and temperature, and 3–5 ratings each of sea spray and visibility were identical between subjects, and ratings differed by at most two categories.
The ‘ride-quality’ item showed sensitivity to acceleration exposure (figure 2), and the fatigue summary score items showed sensitivity to ride quality (figure 3). However, because the response distribution in the fatigue items suggested that a potential floor effect might be present, which could be detrimental to fatigue discrimination, some changes were made to increase sensitivity. The ‘memory’ item, excluded in the validation process based on expert comments—and which nevertheless met the criterion for an acceptable I-CVI—was reintegrated for further evaluation. Moreover, the ‘concentration’, ‘decision’ and ‘memory’ items were revised to accommodate a bipolar response structure (ie, ‘Very high’ to ‘Very low’), and an additional response category was added to both the ‘headache’ and ‘tiredness’ items. Final modifications were also implemented with respect to the musculoskeletal pain items. Feedback from the subjects revealed that they lacked a response option for absence of pain while under pain relief medication; the response structure of the ‘pain event’ item was therefore revised to accommodate this. Finally, the ‘perceived pain cause’ item was removed to reduce the respondent burden.
Sampled acceleration relative to self-reported ride quality for the only three subjects with complete data. Vibration dose value computed as in ISO 2631-1.38
The four top graphs show fatigue-related ratings per ride quality category and the bottom graph shows the number of fatigue symptoms defined as ratings other than ‘No’ for each observation. Figures are based on 58 observations from repeated measurements in seven subjects.
Discussion
This study developed, validated and pilot tested a questionnaire for longitudinal investigation of work exposure, musculoskeletal pain and performance in HPMCP. Ratings from nine experts computed to an S-CVI/Ave of 0.97 and 1.00 for relevance and simplicity, respectively, supported excellent content validity, and the pilot test suggested that the questionnaire, following some adjustments, was feasible for its intended purpose.
The expert ratings supported that the questionnaire content was both relevant with respect to the intended content domains and simple to understand. In the first validation stage the S-CVI/Ave already exceeded the commonly used threshold of 0.9027 28; however, expert item-level disagreement and the multiplicity of comments indicated that further improvements were possible. Items were noticeably refined in subsequent stages, as reflected by the increase in S-CVI/UA, which improved from 0.64 and 0.50 in the first stage to 0.85 and 0.96 in the final stage for relevance and simplicity, respectively, thereby meeting the acceptability criterion of 0.80 for both.27 28 Most expert comments supported that the questionnaire was sufficiently comprehensive. The additional items suggested by three experts were decided against, since they either were indirectly measured or were too peripheral to motivate the additional respondent burden.
Although our content validity indices were exceptionally high in comparison both to our previous questionnaire and to reported results of other questionnaires,18 27 certain adjustments were necessary to finalise the questionnaire. Item 12 (‘headache’) failed to meet an acceptable I-CVI for relevance but was nonetheless retained, as expert comments suggested that this was due to a lack of understanding of its intended purpose as a fatigue summary score item. This decision was supported by the pilot-test results which indicated that it was sensitive to ride roughness. In addition, a potential floor effect detected by inspecting the distribution in fatigue-related items prompted the return of item 15 (‘memory’) and the changes in the response structure of all fatigue-related items.
The chosen item recall periods were in line with general principles of recollection accuracy.23 Frequent everyday events are typically estimated more imprecisely than rare and prominent events,23 which supported a shorter recall period for work exposure and fatigue-related items than for pain-related items. Studies on fatigue recollection suggest that the daily recall bias is within an acceptable level,22 37 whereas studies on pain recollection indirectly suggest that the 7-day recall bias of the pain event itself is within an acceptable level; however, that the pain intensity is systematically slightly overestimated.22 24
While the results from both the validation process and the pilot test supported the adequacy of the questionnaire in quantifying the content domains, it could involve a considerable respondent burden as the final version contains up to 30 items. Upon initial review, the response rate suggested that there was a problem with the feasibility of the questionnaire for longitudinal measurements. The secrecy of the group investigated prevented determination of the exact response rate and attached causes (eg, respondents’ work schedules were classified); however, respondent feedback revealed that they were not allowed to use their cellphones during a 1-week exercise and that two intended subjects did not participate in the marine exercise and therefore dropped out. In addition, Norwegian occupational regulations demand an average 2-day rest per week. Accounting for these factors, we approximated a response rate of >85% for three subjects and 10%–40% for the three remaining subjects in the daily questionnaire section, and 100% for one subject, 50% for three subjects and 0%–25% for two subjects in the weekly questionnaire section. Thus, in this pilot study, half of the respondents had an acceptable response rate for the daily section, but only one of six for the weekly section. Respondent feedback suggested that the low response rate for the weekly section was related to the division of the questionnaire into two parts, and both sections were therefore incorporated into a single web questionnaire. Noteworthy is that in this pilot test, we maximised the respondent burden both in sampling frequency, once following each work shift, and in total questionnaire items. Decrease of either of these two aspects would likely increase questionnaire feasibility for longitudinal investigation.
This study has some limitations. Whereas a large number of experts were included in the questionnaire validation to provide a suitable breadth of knowledge across content domains and to lessen the risk of chance agreement,28 its validity is limited by the proficiency of the expert raters and the consensus panel. Likewise, the results of this pilot test, conducted in a sample chosen to represent HPMCP subjected to the most intense VRS exposure, are limited by the small sample size. With respect to the questionnaire content, performance was indirectly measured via fatigue, as performance and fatigue have previously been associated3 4 33 34 and as performance is hard to capture with self-reported data. To know how far the questionnaire items actually measure performance, it is, however, necessary to link them to objective performance indicators.
The present questionnaire was developed as a complement to the previously constructed questionnaire.18 In conjunction with objective exposure data, the two questionnaires provide a means to quantify the extent of musculoskeletal pain and performance impairments in HPMCP, and to link the contribution of VRS exposure causally to these effects. However, for accurate inferences, their psychometric properties should be further evaluated.
Conclusions
A dynamic web-based questionnaire for longitudinal investigation of work exposure, musculoskeletal pain and performance impairments in high-performance marine craft populations was constructed. Expert ratings supported that the questionnaire content was relevant, simple and sufficiently comprehensive. A pilot test suggested that the questionnaire, following some adjustments, was feasible for longitudinal measurements in the study population.
Supplementary file 2
Acknowledgments
We thank the expert raters for their feedback, and the Norwegian Special Operations Command officers for their participation in the pilot test. We also thank Lea Constan for excellent feedback on the manuscript, Jan Ivar Kåsin from the Norwegian Institute of Aviation Medicine for connecting us to experts and pilot test subjects from the study population, and Stefan Andersson from the Swedish Coast Guard for providing experts from the study population.
References
Footnotes
Contributors KG is leading the research programme of which this study is a part. All authors conceived and designed the study, and constituted the consensus panel. RLM and MPdA outlined the questionnaire and refined it in accordance with the experts’ feedback. RLM implemented the questionnaire online and drafted the manuscript, and MPdA, KG and BOA reviewed and contributed to the manuscript’s development. All authors read and approved the final manuscript.
Funding The Gösta Lundeqvist Foundation for Ship Research (Gösta Lundeqvists stiftelse för skeppsteknisk forskning) and the Swedish Maritime Administration (Sjöfartsverket).
Competing interests None declared.
Patient consent Parental/guardian consent obtained.
Ethics approval The Regional Committee for Medical Research Ethics (Dnr. 2015/576-31), Stockholm, Sweden.
Provenance and peer review Not commissioned; externally peer reviewed.
Data sharing statement No additional data available.