Article Text
PDF
Abstract
Introduction One of the main limitations that can be observed after acquired brain injury (ABI) is the alteration of the awareness of the deficits that can occur in the cognitive skills necessary for performing activities of daily living (ADL). According to the Dynamic Comprehensive Model of Awareness (DCMA), consciousness is composed of offline component, which contains the information stored about characteristics of the tasks and stable beliefs about one’s own capabilities and online awareness, which is activated in the context of the performance of a specific task. The main objective of this project was to generate and validate a detailed cognitive assessment protocol within the context of ADL to evaluate the components of DCMA.
Methods and analysis The proposed protocol consists of two ecological tools: The Cog-Awareness ADL Scale to measure offline component and the Awareness ADL-task: Basic and Instrumental ADL performance-based test to measure online awareness. The aim is to identify the presence of cognitive deficits and anosognosia in patients with ABI within the context of everyday life activities. These two measures will be administered to a group of patients with ABI. In addition, these participants will complete another series of classic tests on anosognosia and cognitive functions in order to find the convergent validity of the two tests proposed in this protocol. The external validity of the Cog-Awareness ADL Scale and the relationships between awareness components within the same ADL domain will be also analysed.
Ethics and dissemination This study was approved by the Ethics Committee of Biomedical Research of Andalusia, on 13 January /2017 (Proceeding 1/2017). All participants are required to provide written informed consent. The findings from this will be disseminated via scientific publication.
Trial registration number NCT03712839.
- stroke
- neurological injury
- adult neurology
This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.
Statistics from Altmetric.com
Strengths and limitations of this study
The protocol presented here raises a global approach to measure the main components of self-awareness related to the execution of activities of daily living (ADL).
The tools designed for this protocol to provide a similar coding system of ADL error types in order to facilitate the comparison and interpretation of potential patterns of relationship or divergence among components.
The online tasks proposed in this protocol include the presence of conflicting/problem-solving situations, thus, all the participants will face the same number of problems, errors to solve and objects to ignore.
This protocol does not aim to assess the information stored in long-term memory about characteristics of the tasks.
Background
Patients with acquired brain injury (ABI) usually suffer severe cognitive deficits that affect their independent performance of activities of daily living (ADL). In some cases, they also show anosognosia, a clinical condition characterised by the lack of awareness of their own cognitive deficits, with a major impact on rehabilitation and patient’s reintegration in the community.1 Indeed, patients with unawareness of their limitations usually produce unsafe behaviours and have a poor treatment adherence, as they believe it is unnecessary, leading to worse functional outcomes and increasing caregiver’s burn-out.2–4
Awareness of cognitive deficits is strictly related to the concept of metacognition: the knowledge about one’s own cognitive functioning. The first theoretical models of metacognition describe it as being composed of different levels, that is, general ‘off-task’ knowledge and specific ‘on-task’ information, and account for a constant flow of information between these components.5 6 Similar characteristics can also be found in several self-awareness models.7–9 According to one of the most comprehensive theoretical models proposed by Toglia and Kirk,9 that is, the Dynamic Comprehensive Model of Awareness, consciousness is composed of two separate components: metacognitive knowledge and online awareness. ‘Metacognitive knowledge’ is the offline component of consciousness and contains the information stored in long-term memory about characteristics and demands of the tasks we usually carry out, how we perform them and the cognitive resources available in order to correctly complete them. This component is formed by almost stable beliefs about one’s own capabilities based on past experiences. The second component, that is, online awareness, is activated in the context of the performance of a specific task. The online component comprises additional subcomponents: ‘anticipatory awareness’, ‘emergent awareness’, ‘self-regulation’ and ‘self-evaluation’ skills. Anticipatory awareness is described as the ability to predict situation demands and one’s own performance in the current task, emergent awareness refers to the process of error detection during the activity and self-regulation is the capability to adapt and adjust the behaviour to correct detected errors. Once the task is completed, a self-evaluation process might begin, by appraising the current performance and comparing it with the expected performance.9 According to this model, offline and online components might dynamically interact and depend on task characteristics such as familiarity, complexity and its value for the person. The prediction of one’s own performance depends on task information stored in metacognitive knowledge and affects the monitoring resources devoted to the task. At the same time, a correct online error recognition might enable the self-regulation process during the performance and a correct self-evaluation following the task. The information obtained during self-evaluation is important in order to change and update metacognitive knowledge if a discrepancy is found between the expected (or anticipated) performance and the actual performance; this process determines the way in which the person will face the same activity in the future.9
Based on this model, one straightforward conclusion is that anosognosia could arise due to different causes. Indeed, a poor error detection ability during the task can lead to the lack of awareness about limitations and problems for task performance.10 11 These authors have found that error detection abilities are strongly related to attention and cognitive control, measured with neuropsychological tests and experimental computer-based tasks. On the other hand, a failure in self-evaluation and updating processes could also lead to anosognosia in patients with acquired brain damage.12 13 In this case, the lack of awareness is expected to be related to memory consolidation impairment, and even when patients are able to detect their own errors during online task performance, such information cannot be integrated with their metacognitive knowledge. Therefore, the patient’s own image clings to the past and the new limitations raised after the damage cannot be incorporated into the metacognitive knowledge, leading to the so-called ‘petrified self’, which has been often described in studies with dementia patients.14 15 Altogether, we can sum up that anosognosia for cognitive deficit after brain damage is a complex phenomenon that might appear due to the alteration of different components. Isolating which components are mostly responsible for anosognosia on a given acquired brain damage patient seems crucial for targeting efficient and personalised rehabilitation programmes.
In order to advance on the knowledge of the different components of anosognosia, the pattern of relationship among them and with other cognitive processes, and its different manifestations in different brain lesioned patients, we need to develop new evaluation tools that provide a comprehensive measure of all the potential components and manifestations of the phenomenon within a given patient. However, as is fully described in the next section, most studies have used isolated measures that focus on a given component (mostly on the offline component), and very few have included a multidimensional approach with intraindividual measures of several components or manifestations. In addition, the tests used to measure different components are usually very different, with the offline component being typically measured with questionnaires about ADL functionality, and the online component being measured with artificial neuropsychological tests or computer-based tasks, where participants are required to answer with simple key press responses according to new and arbitrary stimulus-response mappings.
Assessment of awareness components
Regarding the offline component of anosognosia, its assessment is usually questionnaire based.16 In these questionnaires, the patient and a caregiver answer the same questions about the patient’s difficulties with performance in certain daily tasks or with specific cognitive and emotional functions. The presence of a discrepancy between the two parts of the questionnaire (patient vs caregiver), where the patient under/overestimates his/her difficulties, is considered as an indicator of unawareness of his/her acquired limitations and deficits.1 This kind of measure is useful to evaluate the offline metacognitive knowledge component (in the Toglia and Kirk model), since it assesses the patient’s stable beliefs about his/her capabilities and limitations. These measures are usually the ones considered to make a diagnosis of anosognosia. There are several questionnaires of this sort. Among them, one of the most used is the Patient Competency Rating questionnaire, which determines a cut-off point (>5) in its discrepancy index to establish the existence of anosognosia.1
However, most of these questionnaires do not dissociate among different cognitive manifestations of anosognosia, as they are rather based on global measures of cognitive ADL functioning. Therefore, in order to study whether anosognosia is domain general or domain specific, separate scales are required to be administered to test the self-awareness of different cognitive processes and different ADL.17 Given recent evidence according to which patients can be differently unaware of different cognitive manifestations,18 19 it seems necessary to develop new tools sensitive to several potential sources of deficit in metacognitive consciousness among patients. Even further, global measures of cognitive ADL functioning do not allow studying whether patients are more or less aware of their cognitive deficits when performing basic versus instrumental ADL (IADL vs BADL) or testing anosognosia on severely affected patients who cannot perform complex tasks due to their cognitive and/or physical alterations.
Regarding the measurement of online components of anosognosia, very different strategies and tasks have been used. For example, in metamemory research, online awareness is assessed by asking participants to estimate the likelihood of studied items to be recalled (Judgement of Learning, JOL) or recognised (Feeling of Knowing, FOK) and to rate their confidence about the correctness of a previous response (retrospective confidence judgement, RCJ).6 20 These measures have been adapted and used to assess anticipatory awareness and self-evaluation in different domains by asking participants to predict their performance right before beginning the task and to self-evaluate their performance immediately after the completion of the task. The discrepancy between the patient’s prediction or self-evaluation and the actual performance or therapist evaluation is considered a measure of anticipatory awareness and self-evaluation, respectively. As in the case of JOL, FOK and RCJ, the object of this pre–post evaluation is usually performance in neuropsychological tests (memory, attention, executive functions and perception tests),11 21–25 and very few studies have used ADL tasks19 in populations with ABI.26 In the study conducted with ABI patients, anticipatory awareness was determined by calculating the discrepancy between the evaluations of the therapists about the actual performance and the prediction of the patients before conducting one BADL and two IADL. The discrepancy between the evaluation of the therapists about the actual performance and the patients’ estimation of how they performed the tasks immediately after conducting them was considered as a measure of emergent awareness. As a measure of self-regulation, the authors used the discrepancy between the evaluation of the therapists about the actual performance and the estimation of the patients after a process of joint review (therapists and patients) about the demands of the task, allowing for a review of performance. Other studies have also used ADL tasks (online shopping, using the phone, writing and reading) in other populations of patients.27 28 These two studies found an anticipatory awareness index between the patient’s prediction about his/her performance (difficulty, time, ability for similar tasks) and the actual performance. Moreover, a post or self-evaluation measure was found, comparing the patient’s estimation immediately after the task and the actual performance.
All these studies measured the emergent awareness after the task was completed, whereas error detection and self-regulation (ie, error correction) needed to be measured during the actual performance. In this sense, from experimental psychology, artificial computer-based tasks (ie, Stroop like or go/No go tasks) have been used, in which different measures of online awareness can be obtained. For example, the so-called ‘Post-Error Slowing’ consists in an increment of reaction time after an error is made29 and is considered a self-regulation index, which reflects the adjustment of the performance when an error is detected, in order to avoid another error. Another way to measure emergent awareness is to directly ask the participants to verbalise whether their prior key press response was an error during simple go/no-go tasks.15 30 31 These measures have been recently used to identify impairments in emergent awareness and self-regulation processes in patients with ABI and neurodegenerative diseases.22 32–35
Some of these studies have used multidimensional approaches in order to see whether emergent awareness measured within the context of artificial tasks relate to offline measures of awareness on everyday functioning, with most showing independence among these measures15 (see, ref. 10 for evidence of a positive relationship between metacognitive knowledge and emergent awareness).
One drawback of this approach is that offline and online measures of self-awareness are very different. Therefore, we cannot know whether a lack of relationship between offline and online measures indicates that anosognosia can arise due to different causes or that the performance on artificial computer-based tasks and neuropsychological tests might be different, in terms of processes, from the performance on everyday activities, conferring a low ecological validity to this type of task.36 37 In fact, patients do not usually have previous experience with this sort of artificial tasks and, as referred above, two of the aspects of the task that have been suggested to affect on-line processes are its familiarity and its significance. Therefore, the lack of prior representations in metacognitive knowledge about these artificial task characteristics and demands, and about the correct way to perform it, may reduce the patient’s capacity to correctly predict, monitor and evaluate the quality of his/her performance.9 Therefore, we consider that, in order to directly compare offline and online components of anosognosia within a given patient, it is necessary for both types of measures to ask about similar aspects of daily life.
To our knowledge, very few studies have used direct measures of emergent awareness while patients perform significant ADL. These studies measure the ability of patients to detect and correct their own errors made spontaneously while making a coffee, a toast or a packed lunch.38–42 Most of these studies include a sophisticated error coding system, so that spontaneous errors can be coded as non-detected, detected and detected and corrected, based on participants verbalisations or actions towards correction.38 39 Consequently, they provide a direct measure of emergent and self-regulation components, respectively. Compared with the studies described above, this kind of assessment has a high ecological validity, since highly familiar everyday situations and their characteristics are reproduced in the laboratory.36 43 This allows directly evaluating how cognitive deficits and online awareness component failures affect daily activities in patients, without the need to infer them from artificial tasks.44 However, most of these studies using this ecological perspective to measure emergent awareness and self-regulation have been conducted in populations with neurodegenerative disease38 45 or schizophrenia.46 To the best of our knowledge, only two studies using this perspective have been carried out in patients with acquired brain damage.36 39 Doig et al36 evaluated online emergent awareness in two patients with brain damage during the realisation of several ADL, chosen in collaboration with the patient and his/her family. In addition to the total number of errors made, the authors also measured both spontaneous self-corrected errors and the errors corrected after the therapist offered non-specific or specific prompts (through the ‘pause, prompt, praise’ technique). The other study39 used the Multilevel Action Test to evaluate emergent awareness in patients with traumatic brain injury (TBI). The participants were asked to perform three ADL, such as wrapping a present, making a coffee and preparing a lunch box, but with four difficulty levels, including, for instance, distracting objects and time restrictions. The analysed variables included error detection and correction, as well as a measure of self-evaluation immediately after conducting the task. However, and despite the contribution of these studies to the analysis of this difficult construct, they did not evaluate the offline awareness component (metacognition).
In conclusion, our search revealed that only few studies have provided a multidimensional approach to obtain measures of both online and offline components of anosognosia intraindividually. However, these measures are very different, with the offline component being usually measured with questionnaires about ADL functionality, while the online components are measured with computer-based artificial tasks or neuropsychological tests. Therefore, to date, no study has assessed all the potential components of awareness within the execution of the ADL.
The present study
The main general objective of this study was to generate and validate a detailed cognitive assessment protocol within the context of ADL to evaluate the different cognitive components of consciousness proposed by the Toglia and Kirk’s model (metacognitive knowledge, anticipatory and emergent awareness, self-regulation, self-evaluation and uploading processes). With this protocol, the aim is to analyse these components and understand how each of them can affect ADL performance. The proposed protocol consists of two ecological tools: The Cog-Awareness ADL Scale (ADL scale of metacognitive knowledge) and the BADL and IADL performance-based test (awareness ADL), to identify the presence of cognitive deficits and anosognosia in patients with ABI, always within the context of everyday life activities. One of the main characteristics of these tools is that they are designed to provide a similar coding system of ADL error types in order to facilitate the interpretation of potential patterns of relationship or divergence among them. Second, they are both designed to dissociate among different cognitive processes underlying errors in both BADL and IADL. Therefore, once validated, these tools could be used by future researchers to identify different subgroups of patients with different types of anosognosia.
Thus, one of the main aims of the present study was to test the convergent and external validity of the two proposed ecological tools and to test their diagnostic ability to discriminate between patients with and without anosognosia and from healthy participants. We based the new scale to measure the offline component of awareness on the extended version of the Preliminary Cognitive Scale of BADL and IADL.47 This is an informant-based assessment tool that allows evaluating several cognitive abilities (eg, task schema, error detection, problem solving or task self-initiation in a range of both BADL and IADL). Among other results, previous versions of this scale have shown good psychometric properties for a sample of mild cognitive impairment, dementia and healthy elder participants. A new version of this scale was created and administered for the first time to patients with acquired brain damage, to validate its capacity to measure metacognitive knowledge. To this end, two forms of this version were designed, one to be administrated to patients and the other one to direct caregivers, in order to obtain a discrepancy index. Cognitive items referring to ADL error types reflect the ADL error coding system normally used in online performance-based ADL tasks. Instead of including separate scales to test self-awareness of different cognitive processes 17, the Cog-Awareness ADL Scale helps to measure several cognitive functions and their level of awareness within a single test. Lastly, since the same cognitive items are evaluated in both BADL and IADL, the present questionnaire can identify potential differences in self-awareness when performing simple vs more complex tasks. Moreover, it can be applied more broadly, that is, to patients with brain damage even if at present they cannot perform IADL. We expect to find significant correlations between the discrepancy index of the Cog-Awareness ADL Scale and the discrepancy of the cognitive subsections and of widely studied ADL of classical measures, such as Patient Competency Rating Scale (PCRS). Likewise, we also expect to detect significant differences in this new offline functional awareness scale between patients with offline anosognosia, determined by PCRS (discrepancy index >5). We expect the Cog-Awareness ADL Scale to have a good sensitivity and specificity to determine this tool as a clinical diagnostic measure of offline functional awareness.
The second part of the protocol includes two performance-based ADL tasks (Awareness ADL), one basic (dress-up to get ready to go out) and one instrumental (prepare breakfast with a self-squeezed orange juice and butter and jelly toast), in order to measure the online components of self-awareness. Apart from target objects, highly semantically related but irrelevant objects (ie, distractors) are presented. In addition to distractors, several conflicting/problem-solving situations are included. This type of situations requires interrupting the automated sequence of actions and adding or modifying them to solve these conflicting situations in order to correctly complete the task. The reasons to include this kind of distracting and conflicting situations is threefold. On the one hand, we believe that encountering distractors and conflicting situations makes the ADL tasks more similar to real situations; therefore, they help to increase their ecological validity. Second, the inclusion of highly related distractors and conflicting situations seems to be a promising strategy to identify executive and metacognitive online deficits in patients with brain damage. Favouring this view, previous studies have shown that this kind of distractors have the potential to increase error rate by increasing actions towards them in patients with brain damage,48 dementia and cognitive mild impairment.49 50 In addition, previous work in our laboratory with a simpler version of this task (including the instrumental task only) discriminate between healthy participants and patients with and without anosognosia, constituting a more efficient and simpler way to identify diagnostic differences within ADL tasks, compared with previous coding systems based on spontaneous errors, which is very time-consuming, as it usually requires video recording and a long training process. In addition to measuring the pattern of ADL errors and the ability to detect and correct them in both the BADL and the IADL tasks, the new online test includes measures of additional online components of awareness, such as anticipatory awareness, by asking the participants how they think they will perform the task (after a detailed explanation of the task to be performed, but before the actual performance) and self-evaluation, by asking how they think they did it immediately after conducting the task. The comparison between the pre-evaluation and the actual performance might reflect the ability of the participants to predict their own performance in the context of the specific task, taking in account its particular characteristics and demands (anticipatory awareness). Similarly, comparing the postevaluation with the actual performance allows assessing whether the participant’s explicit self-appraisal of the performance (essential information for the self-evaluation process) is coherent with the reality. Lastly, we will add a final estimation about their own performance 20 min after finishing the task in order to obtain a direct measure of the updating process and comparing immediate self-evaluation abilities and memory consolidation of such events.
With this part of the protocol, we expect to find strong predictions of online measures that have been previously validated and used, such as the Weekly Calendar Planning Activity (WCPA),51 and an online measure based on the phonetic fluency test for the variables of detection and correction of errors made in the task of the awareness ADL designed for this protocol. Likewise, we expect to find relationships between the measures of self-evaluation and short-term memory, and between updating capacity and measures of long-term memory.
As stated in the introduction, the findings about whether the offline and the online awareness components are related to each other are contradictory, probably due to the fact that very heterogeneous tests have been compared. In addition, as far as we know, anticipatory awareness or self-evaluation have not been systematically studied in relation to other components from the model (ie, error detection, error correction or metacognitive knowledge). The protocol proposed in the present study might pose the ideal conditions to further test the Toglia and Kirk’s model regarding the different components proposed, and also to determine the interactions among them, as all measures are taken within the same ADL domain. This is the last aim of this study.
Methods and analysis
Design
Validation of a protocol to evaluate functional anosognosia, analysing both its convergent and external validity through case–control retrospective observation.
Participants
Recruitment
The sample of participants will be constituted by patients with ABI and healthy patients. The group of patients will be recruited from neurological rehabilitation services and associations of people with brain damage of Málaga and Granada, Spain. The group of healthy participants will be constituted by healthy adults of the same age, gender and educational level as the patients of the two groups with brain damage, who will be recruited in the previously mentioned cities via phone call or email. Younger healthy participants are being recruited through online advertisements using an online platform created by the Experimental Psychology Department at the University of Granada. Older participants were recruited either from the researchers and collaborators’ group of friends or relatives, who were verbally informed about the study and invited to receive further information. If they decided to participate, they contacted the researchers by telephone or email to schedule the first appointment.
The occupational therapist in charge of managing the cases will provide the users of these services with a brief summary of the study and will identify possible candidates to participate in it. Those who offer their participation will be asked to provide verbal and written consent and their contact details to allow the coordinators of the project to contact them or their relatives. These, through phone call or email, will be informed about the study in detail and will be appointed for the first evaluation session. At the beginning of such session, the researcher will provide the participants with the sheet containing the information of the project and the consent form. They will be asked to sign and will be registered as participants of the study, so long as they are not excluded after the first screening tests.
Eligibility criteria
The participants will be included in the group of patients with brain damage if they are over 18 years of age and have suffered acquired brain damage (stroke or TBI).
The study will exclude people with severe visuoperceptual deficit, which hinders the execution and completion of the ADL tasks and comprehension deficit, determined by the ability to understand and follow the instructions to complete the Mini-Mental State Examination (MMSE) (in case of doubt, the Boston Naming Test will be administered). The study will also exclude patients with hemispatial visual neglect, evaluated through cancellation tests and bisection lines, measured by a Behavioural Inattention Test,52 and/or motor deficits in both upper limbs, which makes it impossible for the patient to complete the ADL tasks. A cut-off point will be set at 18 or higher in the MMSE to participate in the study.
Evaluation procedure and measures
The professionals of the hospitals and associations will be the ones who will select the patients who meet the established inclusion criteria. Once they agree to contact the researcher, in the first session, after the participants sign the informed consent, the MMSE will be administered to confirm that each of these patients are candidates to enter the study. If a patient is considered a candidate, he/she will be asked to provide his/her demographic data, such as sex, age and education level. For ABI patients, data about aetiology, time since injury and, when available, Glasgow Scale score at admission, will be collected.
After gathering this information, the evaluation procedure will begin, which will be carried out in two or three sessions of approximately 1 hour each (depending on the capacity of the patient). First, the neuropsychological tests and the two offline awareness scales will be carried out. Then, the Awareness ADL will be conducted. The same researcher will carry out the entire evaluation, although blinded in terms of which patient group the participant belongs to, since the family member or caregiver’s evaluation will be archived and the PCRS discrepancy index will not be calculated until the end of this process.
To explain the measures in a grouped manner, we present here three blocks (A, B and C), which correspond to the measures proposed in this protocol, the classical measures of offline and online awareness and the traditional neuropsychological tests.
Ecological consciousness evaluation protocol proposed in the present study
Metacognitive evaluation (offline): The Cog-Awareness ADL Scale
This scale is based on the preliminary and extended version of the Cog-ADL Scale.47 For the aim of this study, there will be two versions of the Cog-Awareness ADL Scale, one to be administered to a direct caregiver and the other to the patient, to observe the discrepancy index in terms of functionality-cognition. First, the scale includes several questions about the patient’s frequency (daily, weekly, sometimes or never) and degree of assistance (totally by him/herself, with little help, with a lot of help and someone does the activity for him/her) with which he/she usually does two BADL and two IADL (personal cleanliness, getting dressed, cooking and managing finances/shopping) before and after the brain damage (items 1 to 16). In the second part of the scale, this tool allows evaluating eight key cognitive aspects related to ability tasks: manipulation difficulties, action schema, distraction, substitution, repetition, error detection, problem solving and task self-initiation in the two BADL and in the two IADL, respectively (34 items). Both patients and caregivers must answer how often the patients present this cognitive-functional error in each of the 4 ADLs: (1) never, (2) sometimes, (3) quite often and (4) always. An example of the items presented in this part is: ‘I get distracted by other objects, touch them or even use correctly some other object in the bathroom that is not necessary for the personal hygiene activity that I am doing at that moment.’ The variables used for the analysis of the Cog-Awareness ADL Scale will be the cognitive-functional errors made by the participants in BADL, on the one hand, and in IADL, on the other hand (see table 1).
Proposed awareness tests created for this protocol and their correspondence to the traditional tests with the aim of finding their convergent validity
Evaluation of online awareness: The Awareness ADL
The performance-based BADL and IADL test is based on tasks designed in our laboratory. According to preliminary data, these tasks are sensitive to the deterioration of executive functions caused by healthy ageing in the everyday context, and they can also be used to identify errors and the alteration of executive processes in patients with anosognosia. The novelty of this study is the addition of a basic task to evaluate the processes of self-awareness in patients with brain damage, as well as the inclusion of measures of the rest of the components of self-awareness proposed by Toglia and Klirk,9 such as anticipatory awareness, self-evaluation and updating.
In this ecological evaluation, the participants will be asked to carry out two ADL: an instrumental activity (making breakfast) and a basic activity (dressing one’s upper trunk). The breakfast task will consist in making orange juice with a teaspoon of sugar and a toast with butter and jam. In addition to the thirteen objects required to complete this task, the participants will be presented with three distracting objects; together, these 16 objects will constitute the set needed to provoke a semantically related action, such as making a cup of white coffee, which the participants will not be ask to perform. All the objects will be randomly deployed on a table for each participant. Furthermore, the task was designed to generate four controlled conflicting situations: (1) the juice maker will be unplugged, with the cable hanging from the edge of the table; (2) the juice maker will be disassembled into three parts, with the container upside down; (3) the toaster will be unplugged, with the cable handing from the edge of the table and (4) the butter knife will be inside the jam jar. Figure 1 shows the details of the objects and their characteristics.
Possible presentation of the different objects in each of the awareness ADL: Orange juice with butter and jam and upper dress task. ADL, activities of daily living.
In the dressing task, the participants will be presented with a basket with different pieces of clothing in it (see figure 1). Next, they will be asked to dress the upper part of their trunk as if they were preparing to go out and imagining it were raining, thus they will have to take the shirt and the oilskin jacket from among the different pieces of clothing. Moreover, they will also find five distracting objects inside the basket: a complete set of pyjamas, a robe, two socks and a clothespin, which they are not supposed to use. As in the breakfast task, four different conflicting situations were designed for the dressing task: (1) the shirt will be presented inside out, (2) with the clothespin attached between the first and second button, (3) the oilskin jacket will be presented inside out and (4) with one of the socks sticking out of the pocket.
To begin the evaluation, the participant will be asked to name all the objects, foods and pieces of clothing presented on the table and in the basket. If the participant omits one of them, this will be pointed out and named for the participant. Then, he/she will be given the following task instructions (breakfast task): ‘I am going to ask you to make a juice with these oranges (for which you will need to cut them first), with a teaspoon of sugar and a toast with butter and jam, as if you were really going to have them, even if you do not right now. If you need motor help, tell me what you need specifically, and I will help you’. For the dressing task, these will be the instructions: ‘Imagine that it is raining right now and we are going to go outside. Please, get the upper part of your body dressed and get ready to go outside in the most appropriately way possible. The task will end when you open the door to go out’. Then, the participant will be asked to answer the anticipatory awareness question, explained in the next section: ‘Could you repeat what I asked you to do?’ If the participant does not repeat all the tasks requested, he/she will be asked again until the participant describes the entire process. Lastly, the participant will be asked: ‘Please, do everything you need to do to carry out the task I asked you to perform. If you need my help you can ask me, although I will not remind you what you have to do while you do the task. Remember that you have to do it as if you were really going to eat it, although you do not have to eat it. Please, let me know when you are done’.
Two variables will be included in the analysis of this measure: detected errors and corrected errors. The types of errors that can be made, which will be coded in this study, are shown in table 1. To evaluate emergent awareness, we will calculate the total of errors made by the patient, both spontaneous mistakes and those caused by conflicting situations and the presence of distracting objects. The detected errors category will be calculated as a percentage, dividing the total errors detected by the total errors made. Table 1 shows a description of how these errors will be coded as detected. The self-regulation processes will be calculated also as percentages, dividing the number of detected errors and conflicting situations by those corrected and anticipated by the participant.
To measure anticipatory awareness in ADL performance, after describing each ADL task and before its performance, the evaluator will ask the participants to rate how well they think they will perform (ie, to predict the number of errors) in a Likert scale (0–4), where 0=‘I won’t be able to do it’; 1=‘I can do it, but I will make many mistakes’; 2=‘I can do it, but I will make some mistakes’; 3=‘I will do it quite well, with few errors’; 4=‘I will do it perfectly without making any mistakes’. This estimate will be compared with the actual performance of the activity. In order to compare it with the participant’s prediction, the objective performance will be measured on a Likert scale (0–4: 0=‘can’t do the task’; 1=‘many errors’; 2=‘some errors’; 3=‘few error’; 4=‘no errors’). The transformation of the absolute number of errors made during the task to a Likert scale will be done by calculating the z score, based on the entire sample, for the total number of errors for each participant and assigning to each range of z score a score of the Likert scale (ie, Z ≤0.5=4 ‘no errors’; −0.5<Z <0=3 ‘few errors’; 0<Z <0.5=2 ‘some errors’; Z>0.5=1 ‘many errors’; if the participant can’t complete the task=0 ‘can’t do the task’). A discrepancy index between the scores of the two Likert scales (prediction minus actual performance) will be calculated to obtain a measure of anticipatory awareness in an ordinal scale ranging from 4 (maximum anticipatory overestimation) to −4 (maximum anticipatory underestimation).To measure self-evaluation, the same Likert scale administered before the task will be administered at the end of the task, although, in this case, the participants will be asked to evaluate the amount of errors made during the task immediately after its completion. The items of this postevaluation Likert scale will be 0=‘I couldn’t do it’; 1=‘I could do it, but I made many mistakes’; 2=‘I could do it, but I made some mistakes’; 3=‘I did it quite well, with few errors’; 4=‘I did it perfectly without making any mistakes’. The self-evaluation index will be calculated as the discrepancy between the participant’s self-appraisal score in the postperformance Likert scale minus the actual performance Likert-scale score, obtaining an ordinal scale, where 4 is the maximum self-evaluation overestimation and −4 the maximum self-evaluation under-estimation. Right after completing the self-evaluation Likert scale, we included qualitative questions about the types of mistakes made, difficulties encountered and strategies used to solve them. Lastly, to measure the updating process, 25–30 min after completing every ADL, the patients will answer the same postperformance Likert scale and a discrepancy index will be calculated as the score obtained at 25–30 min after the end of the task minus the score obtained immediately after each task.
Table 1 shows the variables gathered in this protocol.
Evaluation of other classical measures of awareness previously studied
Metacognitive evaluation (offline): PCRS.1
The primary purpose of the PCRS is to evaluate self-awareness (the ability to appraise one’s current strengths and weaknesses) following TBI. The PCRS is a 30-item self-report instrument that asks the person with brain injury to rate his or her degree of difficulty to accomplish common daily activities in the domains of ADL (eight items), cognitive function (eight items), interpersonal function (seven items) and emotional function (seven items) on a 5-point Likert scale (1=cannot do it; 2=it is very difficult to do; 3=I can do it with some difficulty; 4=it is fairly easy to do; 5=I can do it easily). The subject’s responses are compared with those of a significant person (relative, caregiver or therapist), who rates the subject on the identical items. For the present study, this will be a direct family member or caregiver in every case. Impaired self-awareness may be inferred from discrepancies between the two ratings, that is, when the subject overestimates or underestimates his/her abilities compared with the other informant. A discrepancy index of >5 determines the presence of anosognosia in the patient. This scale has shown excellent correlation in two tests for patients with TBI (r=0.97) and relatives (r=0.92), excellent test–retest reliability (intraclass correlation coefficient (ICC)=0.85) and excellent inter-rater ratings (r=0.92).
In addition to finding the discrepancy index with this measure, the family member’s PCRS will also be used as a measure of functional criteria of each participant, as has been used in a previous study.53
Evaluation of online awareness: the WCPA51 (Short version. Level 1-Version A) (WCPA) and Online Awareness phonetic fluency test
The WCPA is a performance-based test that measures different executive functions, which include: planning, problem solving, inhibition of non-relevant information and maintenance and monitoring of rules. Its short version consists in asking the patient to schedule 10 appointments (which appear in random order) within a 1-week span. Some appointments are incompatible with others, thus the patient must take them into account to plan correctly. In addition, the patient will be asked to comply with five rules, which are explained at the beginning of the test and are kept in view throughout the task. The test provides different variables for its analysis, as well as the achievements when planning, such as the types of mistakes made, the ability to detect them, the time, the rules followed, and the strategies used. The reliability of this test is: ICC=0.65–0.91. Of this evaluation tool, the variables of number of strategies used and self-acknowledgement of errors will be selected.
Online awareness phonetic fluency test. Following the proposal of Robertson and Schmitter-Edgecombe,11 an online awareness measure will be found through the phonetic fluency test of the Controlled Oral Word Association Test (COWAT) to calculate this variable online, dividing the number of errors and perseverations in this task by the total number of attempts.
Table 1 shows more details of the variable types.
Traditional neuropsychological evaluation
The MMSE54 will be included to evaluate the general cognitive state and to establish the inclusion/exclusion criteria. Moreover, the participants’ short- and long-term verbal memory will be assessed with the Rey Auditory Verbal Learning Test.55 Two measures of this test will be used: the number of words recalled after the first time in a free recall test and the number of words recalled in the long-term in a free recall test. As shown in table 2, four neuropsychological tests will be included, which will measure different executive functions; most of these tests have been related to the awareness processes in previous studies. We will administer the INECO Frontal Screening,56 which has proved to be useful to explore several types of executive functions, such as response inhibition and set shifting, abstraction, and working memory. The Colour Trails Test57 is an attention and executive functions test in which the participants have to rapidly connect circles numbered in the correct sequence in the first part of the test. In the second part, the participants must connect numbered circles in the correct sequence, although these alternate between pink and yellow colours. The variables included in this protocol from this test will be the time difference between the completion of part 2 and that of part 1, the error colours in part 2, and the near-misses in the part 2. The Key search test is a short executive function test, which is purported to assess planning and problem solving.32 Finally, we will administer the COWAT, whose objective is the evaluation of Verbal Fluency before phonetic and semantic slogans. Table 2 shows the reliability of each of the tests.
Neuropsychological evaluation tests (and their reliability) to establish the inclusion/exclusion criteria and find the cognitive convergent validity of the awareness ADL protocol
In order to know the general cognitive function level of each participant, and following the proposal of Ownsworth et al58 a neuropsychological functioning composite will be found by calculating the mean of the scores adjusted for age.
Statistical data analysis
A series of a priori analyses were conducted with the programme G×Power to estimate the necessary sample size for the intended analysis.
In all analyses, the alpha value and Power were set at 0.05 and 0.80, respectively. Based on previous research, the effect sizes entered were f=0.47 for analysis of variances (ANOVAs)19 and ρ=0.5 for correlation analysis.41 59 For regression analysis, due to the lack of references for the effect size in the literature, a medium effect size was entered f2=0.15.
The analysis that requires the largest sample is regression: when entering four predictors, a sample size of 85 participants is needed.
Cog-awareness ADL scale: offline awareness
Internal convergent validity of the Cog-awareness ADL scale
The analysis to explore the convergent validity of the Cog-Awareness ADL scale will be conducted within each group separately. First, a zero-order correlation will be performed between the variables of interest of the Cog-Awareness ADL Scale and demographic variables (age, education level), time since injury, severity of damage, PCRS relative’s score and neuropsychological composite. Subsequently, partial correlations will be performed between the measures of the Cog-Awareness ADL Scale, a total discrepancy index, BADL and IADL discrepancy indices and the discrepancy index of each domain of PCRS (ADL, Cognitive function, Interpersonal function and Emotional function), controlling for the demographic, lesion and functionality variables that demonstrated a significant correlation with the Cog-Awareness ADL Scale in the first analysis and correcting the p value for multiple comparisons.
External validity of the Cog-awareness ADL scale
With the aim of determining the external validity of the Cog-Awareness ADL Scale, we will compare the performance of the ABI patients with and without offline anosognosia based on the PCRS discrepancy index (PCRS >5)1 and that of the healthy patients in two the variables of the offline test designed specifically for this protocol (table 1).
The comparison analyses will be conducted using an ANOVA (or analysis of covariance, to control for any demographic, lesion or functioning variable that significantly correlates with the COG-Awareness ADL Scale variable of interest), or a non-parametric test when the data distribution is not normal (Kruskal-Wallis test). If a non-parametric test is required, the method proposed by Martin-Pérez et al60 will be applied, using an unstandardised regression coefficient for these covariates to correct the discrepancy index of the Cog-Awareness ADL Scale.
Effect size estimates will also be calculated through analysis of eta squared (ƞ2),61 when the data distribution shows normality. If the variables do not follow a normal distribution, the Cliff’s delta will be calculated.62
Diagnostic evaluation with Cog-awareness ADL scale
The diagnostic accuracy of the Cog-Awareness ADL Scale will be conducted for the identification of patients with and without anosognosia through a receiver operating characteristics curve analysis, wherein larger areas under the curve indicate better diagnostic accuracy. The optimal cut-off points will be determined by Youden’s index formula, where higher Youden index indicate maximisation of sensitivity and specificity. For each cut-off point, we will calculate the sensitivity (the probability for subjects with the disease to have a positive test), and specificity (the probability for subjects without the disease to have a negative test).
To previously know whether the patient has anosognosia, the cut-off point of reference will be the one proposed by Prigatano et al,1 that is, the discrepancy index of the PCRS of >5 points.
Awareness ADL-task: online awareness
Emergent awareness and self-regulation: internal convergent validity
Due to the absence of a gold-standard test for online awareness, we will include regression analyses in this part of the protocol. In this way, correlations will initially be conducted between the two measures of the Awareness ADL (ie, % detected errors and % corrected errors), the functionality level (the relative’s PCRS), the cognitive functioning (neuropsychological functioning composite), the time since the lesion and the severity of the injury (Glasgow Scale). To find the significance level, the Bonferroni correction will be conducted for multiple comparisons.
The variables that are significantly correlated will be entered as predictor variables, to ensure that they are controlled in the regression analyses. Two analyses will be carried out, one with the dependent variable of percentage of detected errors and the other one with the percentage of corrected errors. The following will be included as exploratory variables: number of strategies used, number of self-recognition errors of the WCPA, online awareness phonetic fluency test and the discrepancy index of the PCRS.
Emergent awareness and self-regulation: inter-rater reliability
Two raters will independently code video recordings of different awareness error categories of Awareness ADL. Inter-rater reliability will be assessed for 20% of the sample, randomly selected. Based on previous studies with this type of methodology49 raters must demonstrate that the intraclass correlation coefficient (ICC) for test reliability is at least 0.90 on all scoring measures. Any disagreements between the coders will be resolved through discussion and reassessment of the videotapes.
Self-evaluation and updating processes: internal convergent validity
The self-evaluation measure will be correlated with the total number of words recalled in the short term in the 5 trials of the Rey Auditory Verbal-Learning Test. The updating capacity index (the self-evaluation score at 30 min after the task minus the self-evaluation score immediately after the task) will be correlated with the long-term variable of the Auditory Verbal Test.
Relationship between awareness components
In order to analyse the relationships between components, a correlation analysis (Pearson or Spearman determined from the data distribution) will be carried out, between the discrepancy index of the Cog-Awareness ADL scale and the errors detected and corrected in the Awareness ADL, as well as with the anticipatory awareness measure, self-assessment and updating. To establish the level of significance, the Bonferroni correction will be applied for multiple comparisons.
Patient and public involvement
There was no patient or public involvement in the design of this study.
Discussion
The present study describes the theoretical and empirical background underlying the need to propose a new protocol of ecological evaluation to identify the presence of anosognosia in people with brain injury through a cognitive and ADL awareness scale (the Cog-Awareness ADL Scale) and a test with performance-based tasks (the Awareness ADL task). The aim of the evaluation method presented in this protocol is to analyse, through different ecological tools (cognitive functional), the different components of awareness proposed by the Toglia and Kirk’s model, in order to identify possible impediments in each of them and understand how each component affects the awareness of the patients on their deficits and performance with ADL.9
According to this theoretical model, awareness consists of two separate components: the offline or metacognitive component, and the online component. The offline component comprises all the information stored in the long-term memory about the tasks we carry out, how we conduct them and the cognitive processes available to complete them. The online component works during the completion of a certain task. This component includes: the capacity to predict the performance of the task to be conducted (anticipatory awareness), with the subsequent preparation that may be involved in this previous anticipation; the detection of errors made during the execution of the task (emergent awareness); the capacity to apply compensatory strategies to correct the errors made (self-regulation); and the ability to evaluate how the task was performed, once completed (self-evaluation).9 These four subcomponents of online monitoring interact with each other to provide the individual with an optimal occupational performance capacity and, in turn, they allow updating the previous metacognitive knowledge (updating) through direct evaluation of the performance of the task that has been carried out.9 According to the model, both the incapability to update the previous metacognitive knowledge and the alteration of the online monitoring processes can lead to the emergence of loss of awareness in people with brain damage.
Therefore, the protocol presented in this study aims to tackle, in an integral manner, all the components of awareness within the execution of ADL. The tests designed for the evaluation of these components pose an improvement with respect to the ones that have already been studied in previous research, since, for instance, the online tasks proposed include the presence of conflicting/problem-solving situations, thus all the participants will face the same number of problems, errors to solve and objects to ignore. This design allows controlling more rigorously the studied variables with respect to the evaluation of the awareness components exclusively regarding the errors made by the participant, whose results may be widely varied. Furthermore, this is the first time that the evaluation of a BADL is included, that is, the dressing task, which can generate a similar number of errors as an IADL.
Additionally, this evaluation protocol will allow understanding the cognitive processes underlying the metacognitive awareness (offline) and the online monitoring during the execution of ADL.
In the future studies, this evaluation protocol will allow classifying patients with ABI and anosognosia in different profiles according to the awareness components affected. In turn, it will be possible to analyse which neuropsychological variables can better predict each of these profiles and their relation with specific difficulties in the execution of tasks performed in a daily context.
As future research lines, the information gathered with this evaluation protocol will allow carrying out specific interventions about the two types of awareness. Regarding online awareness, the finding of results that confirm that certain errors discriminate between groups would allow proposing and designing interventions that gradually incorporate a larger number of distractors that share similarities with the target objects, as well as conflicting situations to be solved, while offering support to anticipate and detect possible errors.
Lastly, it is worth highlighting that including ecological evaluations in the daily clinical practice is very important to any area of health science that attends to people with ABI, such as neuropsychology and occupational therapy. Previous studies with healthy adults have demonstrated that environmental signals are more efficient than knowledge-based mistakes at facilitating the detection of errors.45
Limitations
As a limitation of the present protocol, it is important to highlight that, due to the evaluation time restrictions and the possible tiredness of the patients, only one measure of offline awareness (PCRS) was selected for the convergent validity analysis. Moreover, the measures of anticipatory awareness, self-evaluation and updating are exclusively based on the valuation of the possible errors that will be or were made. Future studies should include other types of factors, such as the time used or the ability to perform similar tasks. Lastly, the present study is focused on the analysis of functional awareness in patients with TBI or stroke, thus these data must be interpreted with caution when generalising to other clinical populations.
Trial status
The evaluation protocol proposed in this study is based on previous tests designed in our laboratory. However, the specific tests and the ADL proposed here are an added novelty. Specifically, the version of the ADL scale presented poses a substantial change with respect to those previously created in our laboratory, in terms of both structure and content. It includes different cognitive items, groups the ADL into two categories (basic and instrumental) and allows obtaining, for the first time, a discrepancy index by adding a scale to be answered by the patient. Regarding the behavioural ADL task, this new version includes new measures, specifically measures of anticipatory awareness, self-evaluation and updating that were not included or analysed in previous studies. Likewise, the basic task (dressing task) is new, and it has only been validated in healthy participants. Lastly, none of the previous studies conducted with the ADL task include a diagnostic validity analysis to determine their potential to become a test. Therefore, we consider that the tests and types of analysis proposed in this study are new and provide multiple additional evaluation elements that have not been previously analysed or published before the submission of the present study. The study is currently in the recruitment phase.
Ethics and disseminations
To carry out this study, we will consider the Good Clinical Practice (GCP) guidelines of the International Conference on Harmonisation, thus guaranteeing the protection of the rights, the safety and the well-being of the participants of the trial in compliance with the principles of the Declaration of Helsinki, as well as the credibility of the data obtained in the clinical trial.
Prior to the first evaluation session, each participant and his/her family will receive the informative sheet and the informed consent (one for the patient and one for the relative) (see online supplemental appendix 1), which will explain the methodology and the development of the study, the voluntary nature of the research and the procedure for the protection of their personal data in compliance with the organic law of data protection LOPD 3/2018, of 5 December. The informed consent obtained from study participants will be written. The data will be kept safe and guarded by researchers from the two universities participating in the study.
Supplemental material
This study was approved by the Ethics Committee of Biomedical Research of Andalusia, on 13 January 2017 (Proceeding 1/2017). The results are expected to be published in scientific journals.
Acknowledgments
We thank CNP for their initial contribution to this project.
References
Supplementary materials
Supplementary Data
This web only file has been produced by the BMJ Publishing Group from an electronic file supplied by the author(s) and has not been edited for content.
Footnotes
Contributors All authors have contributed to designing the tasks presented here to form the evaluation protocol. GR, JAM-B, MR-B and AN-E have written the article and MJFM has reviewed it thoroughly. All authors have read and approved the final manuscript.
Funding This project was funded by the Spanish Ministry of Economy and Competitiveness (ref PSI2016-80331-P) and has undergone peer-review by this funding body. This work also received financial support from University of Malaga and University of Vic - Central University of Catalonia (UVic-UCC).
Competing interests None declared.
Patient and public involvement Patients and/or the public were not involved in the design, or conduct, or reporting, or dissemination plans of this research.
Patient consent for publication Not required.
Provenance and peer review Not commissioned; externally peer reviewed.