Article Text
PDF
Abstract
Introduction As adults age, their senses tend to decline and a large portion of those most affected by sensory decline reside in long-term care. At present, the creation of a sense-sensitive environment in long-term care is a difficult task as there is minimal evidence or tools available to guide this process. The 5Senses screening tool was developed to measure the sense-sensitivity of a particular environment, with a focus on long-term care. The purpose of this paper is to describe a study protocol to assess the psychometric properties of the newly developed 5Senses screening tool.
Methods and analysis We will conduct a psychometric evaluation of the 5Senses screening tool in long-term care based on the Standards for Educational and Psychological Testing Framework. In phase I, we will seek input from international content experts (n=20) to assess the content validity of all sections of the tool. In phase II, we will invite auditors (n=3-9), residents (n=3-9) and staff (n=3-9) to partake in think-aloud sessions to assess response process validity. In phase III, we will conduct field testing of the revised 5Senses screening tool with auditors (n=100), residents (n=100) and staff (n=100) to evaluate additional measures including acceptability, inter-rater reliability, internal structure validity and internal consistency reliability, where possible.
Ethics and dissemination Ethical approval was obtained from the University of Ottawa Research Ethics Board. Findings will be disseminated through a peer-reviewed manuscript, through a dedicated website, through presentations in long-term care communities and through presentations at research conferences.
- psychometric testing
- senses
- screening tool
- long-term care
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The psychometric methods used in this study will ensure valid and reliable research findings regarding the sense-sensitivity of long-term care facilities.
These methods will help further refine the screening tool and inform future studies to conduct further testing in other settings.
This study may allow researchers to study the impact of the sense-sensitive environment on resident outcomes.
The measure of ‘relation(s) to other variables’ will not be evaluated in this study because we found no other tools that measure the five senses constructs.
Very few items on our tool are supported by randomised controlled trials and therefore most items are based on untested empirical evidence. Further testing of these items are needed.
Background
The five senses (ie, hearing, vision, smell, taste and touch) are one of the key means by which individuals interpret and interact with the surrounding environment. Sensory abilities diminish with age. Around a third of those over the age of 65 have disabling hearing loss.1 Serious vision impairment is also more prevalent with age, with 81% of all cases seen in those 50 years or older.2 The sense of touch deteriorates at a rapid pace, with around 1% of tactile sensation lost per year from 18 years onward.3 Excluding the many cases of chemosensory disorders (eg, loss of taste or loss of smell) that go unreported due to the subtle nature of the problem, age-related sensory impairments are increasing globally.4 Considering that a large portion of those affected by sensory decline reside in long-term care (LTC) settings, the design of these facilities to accommodate these deficiencies is a topic worth exploring. At present, the creation of a sense-sensitive environment within LTC is a difficult task given that there is minimal evidence or tools available to guide this process.
A recent systematic review by Elf et al 5 found that the instruments (n=23) developed to assess the physical environment in healthcare varied in their comprehensiveness of psychometric testing. Although several instruments were developed to assess the physical environment in LTC facilities,5–9 none have looked comprehensively at the five senses. For example, both the Therapeutic Environment Screening Survey for Nursing Homes7 and the Environmental Audit Tool8 have some items related to the five senses (ie, odour, lighting), however, these instruments are observational tools and do not include an all-inclusive list of items related to each of the five senses. More recently, another tool called Dining Environment Audit Protocol9 incorporates some important items related to the five senses, however, it is specifically focused on the dinning environments in LTC facilities. With the increased need for an age friendly environment in LTC facilities, an in-depth focus on assessing the five senses is essential and this focus must include considerations about the environment, the perspectives of staff and most importantly, the perspectives of residents.
LTC settings that support and adopt interventions to enhance sensory capacities will excel in creating a stimulating and enriching environment for their residents. The 5Senses screening tool was designed to assess current sense-sensitive practices and also to serve as a tool to help identify opportunities for improvement in any LTC setting regardless of its current state. The purpose of this paper is to describe a study protocol to assess the psychometric properties of the newly developed 5Senses screening tool.
Development of the 5Senses screening tool
The development of the 5Senses screening tool was guided by ecological model of Lawton and Nahemow.10 Older adults are particularly sensitive to their environment as a result of the increasing number of sensory impairments that result from old age.10 According to the ecological model, in order to maintain independence and quality of life, an older adult’s capacity and demands on the environment must be in congruence.10 Lawton et al argue that the physical environment has the potential to enhance or constrain an individual as they age.11 Therefore, the physical environment must be adjusted to suit the needs of individuals in the later stages of their life.10
The 5Senses screening tool was developed to measure the sense-sensitivity of a particular environment, with a focus on LTC. The practices identified in the screening tool are based on the results of a systematic review.12 In this review, we collated all current knowledge examining the relationship between the sensory practices and the quality of life of residents living in LTC settings. The search strategy yielded 4166 articles of which 123 articles were eligible for full-text screening. A total of 38 studies were included in the final data extraction. The most common interventions reported were related to: touch (n=15), vision (n=10), smell (n=2), hearing (n=2) and more than one sense (n=9). No studies were found on taste (n=0). Examples of interventions reported in the included studies were: sound and music, multisensory environments, aromatherapy, therapeutic touch, pet therapy, massage, lighting therapy, gardens and use of art or pictures. The results of this systematic review were used to assist with the development of the 5Senses screening tool.
The key principles of the tool were that it had to (1) be comprehensive in scope, (2) provide simple ways to action and prioritise the gaps discovered and (3) be user friendly. The 5Senses screening tool assesses the sense-sensitivity of the environment, and captures the related organisational policies and procedures as well as resident and staff perspectives about the sense-sensitivity of the environment. The 5Senses screening tool guides the user through the following steps: a walkthrough of the environment (section 1—facility level), a review of the policies and procedures (section 2—facility level), an assessment of resident perspectives (section 3—resident version) and an assessment of staff perspectives (section 4—staff version). The results from these steps are then used to determine the level of sense-sensitivity of the environment.
An initial pretest of the 5Senses screening tool was conducted in two (n=2) LTC facilities in the USA. The purpose of the pretest was to determine preliminary content validity of the tool, to determine ease of administration, to identify any items that were not worded clearly, to determine adequacy of instructions and to determine the general flow of the tool. As part of the initial pretest, three (n=3) focus groups were conducted in each of the two (n=2) facilities. Participants were recruited from each of the LTC facilities for the pretest. The first being located in New Paltz, New York, and the second located in Collingswood, New Jersey. Focus groups were organised according to roles, with the first group consisting of administrators (including directors of care and facility managers), the second group consisting of care staff and support staff (including registered nurses, allied health professionals, personal support workers, activity organisers, catering providers, cleaning staff and maintenance workers) and the third group consisting of residents and families. A total of 45 participants contributed to the focus groups. No inclusion or exclusion criteria were applied for participation in the focus groups and involvement was voluntary. Participants worked in a variety of roles within the LTC facility and had worked in these roles for varying lengths of time, which enhanced the data collected during the pretest.
Participants were given the opportunity to review the tool in advance of the focus groups to allow them time to examine the document. The pretesting identified item wording and terminology that needed further clarification as well as items that should have been included in the tool, adequacy of instructions and the general flow of the tool. Participants were also asked to consider the logistics of completing the tool, including the time to perform the audit and who should be involved in the process. Based on feedback obtained in the pretest, revisions were made to the tool. The content and proposed scoring for each section of the tool are listed in table 1.
5Senses screening tool sample content and proposed scoring
Proposed scoring
In sections 1 and 2, each sensory item is scored in a binary fashion with ‘yes’ answers being assigned a value of ‘1’ and ‘no’ answers being assigned a value of ‘0’. In sections 3 and 4, each sensory item is scored using a Likert scale ranging from 1 to 5. For individual item scoring, we will calculate the mean for each item. For domain scoring (if feasible), we propose to calculate the ‘mean of means’ for each domain score. The proposed scoring will be further refined based on the psychometric findings (ie, factor analysis, item total) from this proposed study.
Methods
Beginning in March 2019, we will conduct psychometric testing of the 5Senses screening tool in three phases based on the Standards for Educational and Psychological Testing Framework.13–15 In phase I, we will assess the content validity of all sections of the tool (sections 1–4). In phase II, we will assess response process validity (sections 1–4). In phase III, we will evaluate acceptability, and inter-rater reliability (sections 1–2), and we will evaluate acceptability, internal structure validity and internal consistency reliability (sections 3–4).
Resident and public Involvement
Residents were not involved in the development of the protocol.
Phase I: content validity
Content validity refers to the degree to which the items on the tool represent the content of interest.16–18 International content experts from LTC will be invited to evaluate the 5Senses screening tool for relevance, clarity and missing items through an online survey.
Settings and participants
We will seek international representation from key experts (n=20) comprising the following: (1) administrators, directors of nursing, facilities, dietary services and researchers; (2) residents including advisors on resident and family councils and (3) staff working in LTC for >2 years. It is recommended to have a minimum of five experts for content validity testing.19
A list of experts obtained from existing contacts and from an internet search of international relevant organisations will be compiled including, but not limited to the Global Ageing Network, the International Longevity Centre Global and the WHO Department of Ageing and Life-Course. Participants will be invited by email to participate and will provide implied consent by completing and submitting the online survey.
Procedures
The experts will be asked through the online survey to evaluate and comment on the relevance of each item of the 5Senses screening tool in relation to their experiences of sense-sensitivity in LTC. Specifically, experts will be asked to rate the relevance of each item on a Likert scale: 1 (not relevant); 2 (item needs some revision); 3 (relevant but needs minor revision); and 4 (very relevant) as well as provide an explanation for their decision.16 Experts will also be able to add their suggestions on any missing items.
Data analysis
Each item rating on the tool will be averaged. We will also calculate the Scale-Content Validity Index for each domain. Based on the feedback from the experts, we will revise the tool as necessary, prior to phase II.
Phase II: response process validity
Response process validity involves verifying the fit between each question on the tool and the responses provided by the participants.16–18 We will conduct think-aloud sessions in an iterative fashion to obtain verbal feedback from participants on the proposed workflow and their experiences with the 5Senses screening tool.20 This approach will aid in understanding whether participants are interpreting the tool the way it was intended, which helps to ensure that participants interpret and respond to survey items in the manner intended by the researchers.
Settings and participants
We will conduct two to three rounds of think-aloud sessions with LTC facilities in Ottawa, Canada using a convenience sample. Participants will consist of auditors/designated leads from the LTC communities (n=3-9) for sections 1 and 2, residents (n=3-9) for section 3 and staff (n=3-9) for section 4. We will recruit participants through our primary contact in each of the LTC facilities. The inclusion criteria will include: (1) auditors: administrators, directors of nursing, facilities and dietary services who work full-time or part-time at one of the facilities; (2) residents including advisors on resident and family councils and (3) staff working in LTC for >2 years. Participants will need to be fluent in English. The research assistant will provide information about the study and obtain informed consent from the potential participants.
Procedures
Participants will be invited to participate in 60–90 min individual audio-recorded think-aloud sessions, which consists of going through the tool while explaining his or her interpretation of each question on the tool. We will make revisions following the first round of think-aloud sessions, and proceed to the second and/or third round in an iterative manner until no further changes are required.
Data analysis
The think-aloud sessions will be recorded and transcribed verbatim. Following each round, two researchers will independently analyse the transcripts using content analysis to provide a comprehensive and accurate descriptive summary of the participants’ perspectives.21 Discrepancies will be reviewed and resolved by a third researcher. Findings from the content analysis will then be used to revise the tool between each round and for the field testing.
Phase III: additional psychometric testing
We will conduct field testing to collect evidence on: (1) acceptability which refers to the ease of use of the tool17 (sections 1–4); (2) inter-rater reliability which refers to the likelihood that an audit conducted by two individuals will produce the same results (sections 1 and 2); (3) internal consistency reliability which refers to the correlation between the items that make up the overall score for that domain16–18 (sections 3 and 4) and (4) internal structure validity which examines the relationships between groups of items16–18 (sections 3 and 4).
Settings and participants
Approximately 50 LTC communities will be invited by email to participate. We will select LTC communities of different sizes (small <25 beds, medium (26–100 beds) and large >100 beds), which provide 24 hours nursing care with the help of our international experts. From each of these LTC communities, we will recruit approximately two auditors (n=100) for sections 1 and 2, two residents (n=100) for section 3 and two staff (n=100) for section 4. For psychometric testing studies, it is recommended that approximately 5–10 participants be recruited per item on the tool22 23 with a minimum of 100 participants for best results.22
For this phase, the inclusion criteria will be the same as in phase II with the addition of participants being able to access the tool electronically through a weblink. Participants in each LTC community will be approached by the LTC administrator or delegate, and those who express interest in participating will be contacted by the research assistant to who will provide further information about the study and obtain informed consent.
Procedures
For sections 1 and 2 of the 5Senses screening tool, a training manual and a set of training slides will be created and distributed to all study sites. The training will consist of viewing the training slides as well as a guided discussion on individual questions and response categories. All auditors will be trained prior to the start of data collection. Two auditors will independently and concurrently complete the tool in each LTC facility.
For sections 3 (residents) and 4 (staff), residents and staff will be provided with the weblink to complete their respective survey online. All data will be collected using survey software (SurveyMonkey, San Mateo, California, USA, http://www.surveymonkey.com). We will seek institutional review board approval at each site (if applicable).
Data analysis
Descriptive statistics (ie, frequency, mean, SD, maximum, minimum and mode) will be used. For acceptability, we will examine the frequency of missing data. We will also assess the time it takes to complete each component of the tool.16–18 For any missing data, we will also follow-up directly with the auditors to gain more insight into why items were not completed.
For reliability, we will use the following tests to examine inter-rater reliability for categorical items: (1) percentage of agreement and (2) weighted kappa.17 To assess internal consistency reliability, we will use the following coefficients, or estimates of between-score correlation: (1) Cronbach’s α for randomly equivalent measures, (2) Guttman split-half for parallel measures and (3) Spearman-Brown for any changes in the number of items on the tool.17 Internal consistency coefficients can range from 0 to 1. A coefficient of 0.70 is acceptable for newly developed scales such as the present tool under study.24
For internal structure validity, we will conduct an exploratory assessment rather than confirmatory in nature since this will be the first field assessment of the 5Senses screening tool. Therefore, to examine the underlying dimensional structure of the multi-item domains contained in the tool, we will conduct: (1) item to total correlations, (2) item-total statistics and (3) principal component analysis (PCA). Any missing values will be treated as such with no substitution or imputation. From the item to total correlations, any item that correlates with its scale (domain) score below 0.30 will be discussed and revised.25 From the item-total statistics, items that, if removed, cause a substantial change in the domain’s Cronbach’s α score will also be discussed and flagged for potential revision. For the PCA, a Varimax rotation with Kaiser normalisation will used to enhance its interpretability. Factors (domains) will be identified using the standard 1.0 eigenvalue cut-off rule and visualisation of scree plots. Items with factor loadings ≥0.35 will be retained.26 Items that cross load (ie, factor coefficients ≥0.35 on two or more factors) will be flagged for discussion and possible revision.
Ethics and dissemination
Ethical approval was obtained from the University of Ottawa Research Ethics Board. Findings will be disseminated through a peer-reviewed manuscript, through a dedicated website, through presentations in LTC communities and through presentations at research conferences.
Discussion
This paper presents the protocol for the psychometric testing of the newly developed 5Senses screening tool. The psychometric assessment of the 5Senses screening tool will help to ensure that it provides valid and reliable research findings regarding the sense-sensitivity of LTC facilities. The use of this tool will provide key information about the practices, or lack of practices that exist related to the five senses. The findings from the audits will highlight gaps in the sense-sensitive environment of LTC settings and will guide organisations to improve their sense-sensitivity. Further research on the effects of the five senses on quality of life is needed.
During our literature review, we found no other tools that measure the 5Senses environmental constructs, making it impossible to evaluate the tool on the basis of its ‘relation(s) to other variables’, as this metric is aimed at comparing the current 5Senses screening tool against a similar tool which has demonstrated reliable and valid results. Also, very few items on our tool are supported by randomised controlled trials and therefore most items are based on untested empirical evidence. Further testing of these items are needed. We anticipate that this tool will be useful as a stand-alone tool as well as in conjunction with other environmental assessments.
Furthermore, this screening tool is meant to provide a general measure of the sense-sensitivity in the environment, and may not consider every resident’s individual preferences in what they consider a stimulating environment. However, LTC communities can chose to administer the resident and staff sections of the tool to all or to a large sample of their residents and staff to obtain the overall perspectives on the sense-sensitivity of the environment. Additional qualitative information may also need to be gathered before any significant changes are made to the environment. Agitated people or people with advanced dementia may be better served in a peaceful environment with fewer stimuli. The resident and staff sections of the tool allow the possibility to gather data from all residents and all staff in the home in order to capture the different perspectives and needs for that particular LTC community.
References
Footnotes
Contributors CB and JS were both major contributor in the study conceptualisation and writing the manuscript. Both the authors read and approved the final manuscript.
Funding Preliminary work was previously funded by a research partnership between Sodexo Quality of Life Services and the University of Ottawa LIFE Research Institute. The development and testing of the tool described in this protocol is supported in part by the Bruyère Centre for Learning, Research and Innovation (CLRI) in long-term care.
Disclaimer Bruyère CLRI was not involved in the design of the study and in writing the protocol manuscript.
Competing interests None declared.
Ethics approval Ethical approval was obtained from the University of Ottawa Research Ethics Board.
Provenance and peer review Not commissioned; externally peer reviewed.