Assessment of Prospective Memory –  a Validity Study of Memory for Intentions Screening Test

Overview

Aim:
The goal of the present study was to validate the Czech version of the Memory for Intentions (Screening) Test (MIST, 2010). We included standardized testing material, translation of administration and scoring, and assessment of normative data for the MIST in the Czech population.

Introduction:
Prospective memory (PM), i.e., the ability to remember and implement intentions after a delay, is essential as a subsystem of episodic memory for the maintenance of independence and execution of activities of daily living. PM assessment thus plays an important role in the dia­gnosis of episodic memory disorders. However, there are currently no standardized and validated PM tools in Czech language.

Methods:
The Czech version of MIST was administered to 30 healthy persons.

Results:
The MIST Summary score correlated at a medium level with a range of neuropsychological measures including memory retention, mental flexibility, and resistance to interference (all rho = 0.37– 0.42; all p < 0.05). The reliability of MIST in terms of internal consistency was insufficient when analyzing the eight individual MIST trials (α = 0.50), as was split‑ half reliability (split‑ half reliability = 0.56). In contrast, there was a high degree of reliability between six subscales classified by type (delay, cue and mode of response; α = 0.88, split‑ half = 0.95).

Conclusion:
The reliability and validity of the Czech version of MIST is comparable to the original English version. The study opens access to standardized PM assessment in clinical populations in the Czech Republic.

Key words:
Memory for Intentions (Screening) Test – prospective memory – declarative memory – validity – reliability – episodic memory

Introduction

Prospective memory (PM) refers to the ability to carry out intentions after a delay; for instance, to phone someone at a particular time in the future. PM is a subsystem of declarative, and in particular, episodic memory [1– 4]. It is theorized to be comprised of a number of cognitive processes, in particular intention formation, intention retention, intention initiation and intention execution (Fig. 1) [5– 7]. The interval between the formation and execution of an intention may last minutes, hours or days. On functional imaging (fMRI), PM tasks have been shown to primarily activate the prefrontal cortex (Brodmann area 10) and the medial temporal cortex (hippocampal formation) [8,9]. The successful performance of PM tasks requires the ability to initiate and plan an action, inhibit ongoing activ­ities and identify a cue for the task to be carried out in the future (Fig. 1) [10,11].

Early studies on PM explored common and distinctive characteristics with respect to retrospective memory [12]. Contemporary research has placed great­er emphasis on the critical role of PM in the preservation of functional independence, including instrumental activities of daily living (IADL) such as adhering to medication regimes, managing finances, grocery shopping, housekeeping and employment [13]. Considering that a patient with­out intact PM cannot be relied upon to take their medication at the right time and in the right quantity, the construct of PM and its assessment may have great clin­ical utility. Thus, the evaluation of PM may yield knowledge of immediate value in clinical practice.

Although the integration of PM test­ing into comprehensive memory testing would be beneficial, PM testing tools are rarely reviewed and their use in clinical practice remains limited [14]. A survey by Rabin et al (2005) determined that the Rivermead Behavioral Memory Test was the only validated tool among memory tests in use to include a specific index for PM [14,15]. As stated by Woods et al [16], this surprising fact may be due to the small number of tools for PM measurement that have been standardized and have an analysis of psychometric characteristics and normative data. However, two standardized tools for PM measurement have recently been published: the CAMbridge PROspective Memory Test (CAMPROMT; [17]), and the Memory for Intentions (Screening) Test (MIST; [18]). Currently, there are no practical, standardized tools for PM measurement available for use in the Czech Republic, only unvalidated experimental techniques [19]. Moreover, there are no comprehensive descriptions of PM in the Czech literature [20].

MIST was developed and introduced by Sarah Raskin and Carol Buckheit in 2004. A few years later, in 2010, an English version was standardized and validated on an American population [18]. Several studies have described the psychometric characteristics of MIST, such as reliability (internal consistency, inter‑ rater and test‑ retest reliability) and validity (content and convergent validity) [16,18,21,22]. These stud­ies have shown that PM correlates with exec­utive function, verbal work­ing mem­ory and retrospective memory. Regard­ing the relationship between executive function or working memory and PM, Schitzspahn et al (2013) reported that inhibition and shifting appeared to be essential aspects of cognitive control involved in PM performance in young and old adults, whereas working memory was not revealed to be a significant predictor of PM performance [13]. Various studies have documented discriminatory validity in clinical cohorts. For example, MIST has revealed evidence of impaired PM in patients with Alzheimer’s disease, Parkinson’s disease, and brain injury [23– 29]. Carey et al (2008) demonstrated the ecol­ogical validity of MIST in predicting IADL deficits in HIV‑ positive subjects [23], and MIST has been shown to predict medication adherence in individuals with schizophrenia [24]. An overview of all studies concerning MIST validity in clinical cohorts is available in the Raskin et al study [18].

MIST has proven to be a robust tool for the measurement of the PM construct in a number of studies [17,18,22, 24– 26,28,29], however, it is still unavail­able to Czech psychologists. Hence, the primary objective of the present study was to develop a version of MIST for use in the Czech population. Specifically, we endeavored to convert the test to a Czech version (translation and back‑ translation), standardize test materials, validate the basic psychometric characteristics (demographic factors, inner consistency and split‑ half reliability) on sample of the Czech population and describe its construct validity by correlation with estab­lished gold‑ standard tests.

Patients and methods

Participants were recruited through flyers and advertisements from the general com­munity and afterwards using snowball sampling. We then obtained a brief medical history of each subject via telephone (OB). Subjects meeting the inclusion criteria were then tested (OB). A cohort of 30 healthy subjects were included in the study and met the following criteria for enrollment: interviews excluded all participants with a history of head trauma with loss of consciousness, cerebrovascular accident, abuse of alcohol or other psychoactive substances, and individuals with a history of neurological or psychiatric disease (e. g., epilepsy, multiple sclerosis, schizophrenia or ongoing delirium). We additionally excluded persons cur­rently undergoing radio‑  or chemotherapy or with serious internal diseases, myocardial infarction, diabetes mellitus, etc.) or with sensory deficits. Participants meeting the above criteria were then tested for cognitive efficiency, manifestations of depression and activities of daily living (ADL). In order to prevent the inclusion of subjects at risk of developing neurodegenerative disease, limits for enrollment were set on the Mini‑Mental State Examination (MMSE) [30] at a value of ≥ 26 points, the Beck Depres­sion Inventory (BDI‑ II) [31] at a score of < 13, and the Functional Activities Questionnaire (FAQ) [32] at ≤ 4 points. They underwent a comprehensive neuropsychological assessment as part of a research project supported by Charles University Grant Agency. Demographic characteristics of the cohort and their basic functional characteristics are presented in Tab. 1. With all conditions met, we administered a comprehensive neuropsychological battery including MIST (Tab. 2). The test battery was divided into several cognitive domains according to the classification of mental function. The study was approved by the local ethics committee and all participants provided signed, informed consent. All the tests were administered under standard neuropsychological laboratory conditions and were conducted by single, trained psychologist (OB). The author of the Czech translation (OB) had the permis­sion of the license owner (Psychological Assessment Resources) to use the test in the Czech population and to translate the original to Czech. Back‑ translation was done by a translation agency, and comparison of the original and back‑ translation by the original author (SR) and the author of the translated version (OB) was performed.

MIST test construction

MIST contains a set of eight PM trials. The test was constructed to satisfy the accept­ed criteria of a PM task [3,6,33]:

• a) intentions are supposed to be realized after a delay,
• b) there is a separate ongoing activity (attention distraction task),
• c) the test provides a limited period of time after which the intention has to be retrieved from memory and implemented.

The duration of the test was approximately 30 minutes, during which the subject was administered an ongoing, attention‑ distraction task (crossword puzzle). All of the MIST tasks were divided in a bal­anced way between:

1. delay interval (either 2 or 15 minutes),
2. cue type (either time‑based, e. g., “tell me in 15 minutes time that we should have a break”, or event‑based, e. g., “when I pass the red pen to you sign your real name on the paper”),
3. mode of response (verbal, such as in the first example, or activity‑related such as in the second example), which may then be used in choosing a strat­egy for rehabilitation [22].

The Retrieval index was constructed according to Carey et al [23], (Tab. 3). A 24- hour delayed recall followed completion of the eight items in MIST to determine how many hours the subject had slept the day of testing (a task designed to simulate everyday life). The ongoing activity was a Word Search Form (range 0– 40 words). Following completion of the eight recall tasks, recognition items consisting of three multiple choice items and were scored (right answer out of three). All tasks were related to everyday activ­ities so as to have as much ecological validity as possible [22,34]. MIST also allows for error analysis regarding the cause of PM failure [35], which was recorded with the aid of a qualitative scoring system designed to differentiate six types of errors:

1. failure to remember that there was an intention (to realize the intention),
2. loss of content error (substitution of the target activity by an alternative activity at the correct time, or failure to recall the content at all, e. g., remembering at the correct time that “something” should have been done),
3. loss of time error (executing the correct intention at the wrong time),
4. task substitution (e. g., mistaking a verbal response for an activity‑based response or vice‑ versa),
5. place losing omission error (completing only part of the task or repeating the previous one),
6. random error (errors that did not fit into any of the previous categories).

Each of the tasks was related to the six subscales of MIST (time of delay, type of cue, mode of response, each was rated from 0 to 8 points), each containing four particular tasks for PM. Their sum was rep­resented by a summary PM score (ranging from 0 to 48 points). Recognition was a forced‑ choice task between three alternatives (total 0– 8 points). The 24– hour task (range 0– 2) consisted in sending a text message to the administrator within a precisely defined interval (9– 10 a. m.), with the subject allowed to use any strategy to initiate intent retriev­­al (e. g., diary) without being explicitly instructed to do so during testing.

A comprehensive neuropsychological battery was administered and consisted of tests for reading (National Adult Reading Test in the Czech version, CART [36]) as well as general cognitive performance (MMSE [30]; sustained attention: Trail Making Test, part A (TMT‑ A [37]), Digit span from WAIS‑ III [38], Spatial span from WMS‑ III [39]; Working memory: Digit span backwards from WAIS‑ III [38], Spatial span backwards from WMS‑ III [39], Letter‑ Number Sequencing from WMS‑ III [39]; executive function: Tower of London (TOL [40], verbal fluency (letters N + K + P [37]), Trail Making Test, part B (TMT‑ B [37]), Wisconsin Card Sorting Test for the number of perseverative responses (WCST‑64 [41]), Victoria Stroop Test, interference condition (VST [42]); speech: Similarities from WAIS‑ R [43], semantic fluency (animals + clothing + shopping [37]); mem­ory: Rey Auditory Verbal Learning Test (RAVLT [37]), Family Pictures from WMS‑ III [39] and Buschke’s Memory Capac­ity Test (MCT [44]); visuospatial abilities: Benton Judgement of Line Orientation Test (BJOL [45]); manifestations of depression: Beck Depression Inventory (BDI‑ II [31]); activities of daily living (FAQ [32], see Tab. 2).

To assess normality, we examined Q‑ Q plots and performed the Kolmogorov‑ Smirnov test. Given that deviation from normality was detected, we used the Spearman rank correlation coefficient (rho) to evaluate correlation between MIST and the demographic vari­ables age and years of education, as well as other cognitive measures. In accordance with convention, the strength of correlation was rated as low (rho = 0.10– 0.29), medium (rho = 0.30– 0.49) or high (rho = 0.50– 1) [46]. For demographic variables we used the nonparametric Mann‑Whitney U test. To analyze reli­ability, the Cronbach α was determined in the case of internal consistency and the Spearman‑ Brown formula in the case of split‑ half reliability. The magnitude of effect in nonparametric tests (r) was estimated according to Cohen’s effect sizes [47] as small (> 0.1), medium (> 0.3), or large (> 0.5). The level of statistical significance was set at α = 0.05. All presented analyses were performed using IBM SPSS Statistics software (Version 22.0, SPSS Inc., Chicago, IL, USA).

Results

Descriptive characteristics of the cohort are presented in Tab. 1. Descriptive cha­­­-r­acteristics of MIST performance are pres­ented in Tab. 3. Deviation from normal­ity was detected in the distributions of all MIST variables with the exception of Word Search Form (p = 0.36). To evaluate correlation between demographic vari­ables and MIST performance, we therefore applied the nonparametric Spearman correlation coefficient (rho), the results of which are shown in Tab. 3. With regard to several significant correlations between MIST indices and gender (Tab. 3), we looked for any gender‑related performance differ­ences. The Mann‑Whitney U test revealed significant gender differences for the MIST Summary score (men: Md = 38.8, n = 17; women: Md = 42.5, n = 13; U = 62, z = – 2.1, p = 0.038, r = 0.38). Additionally, positive correlation was detected between MIST and demographic variables with respect to age (recognition rho = 0.52) and education (time cue and re­cognition rho = 0.42– 0.43).

Analysis of inter‑ item consistency of the eight individual MIST trials revealed Cronbach α = 0.50. Internal consistency of six of the MIST subscales classified accord­ing to response modality (delay, 2 versus 15 minutes; response type, verbal versus action; time‑based versus event‑based cue) resulted in Cronbach α = 0.88. The split‑ half reliability of eight MIST trials adjusted according to the Spearman‑ Brown formula = 0.56; that of six subscales = 0.95.

Regarding construct validity, Tab. 2 shows the correlation between MIST and the items in the comprehensive neuropsychological battery. MIST correlated most highly with RAVLT delayed recall, then with TMT‑ B, TOL, Spatial span forward, CART and MCT. The effect sizes of correlations were determined as medium‑ large and the differences between particular correlation coefficients were small (rho = 0.37– 0.42).

Correlations between all the MIST indices are presented in Tab. 4. The MIST Sum­mary score correlated with other MIST indices (rho = 0.58– 0.97), with the exception of the 24 hour delay item, recognition task and interfering task (Word Search Form). Overall, the error scores were highly correlated with the Summary score, although in some error subtypes no correlation was detected (task substitution, random error), in one case due to zero variability in all subjects (PLO).

Our study may serve as a source of normative PM performance in the Czech version of MIST. As the above correlation analyses suggest, the MIST Summary score is independent of age and education, though it is dependent on gender (as men and women differed from one an­other significantly). The distribution of that score cannot be regarded as meeting the demands for normality (see normality analysis). Hence, it is impossible to construct a regression equation for analyzing the effect of gender on MIST performance. For tentative estimation of MIST performance in healthy persons (age range 38– 81 and years of education 11– 19) we recommend using Md (median) values (women Md = 45.0, men Md = 39.0), or descriptive characteristics separately for women (M (mean) = 42.5 ± 6.2 SD) and men (M = 38.8 ± 5.7 SD). The average duration of MIST administration was 24.2 ± 1.7 minutes, which is comparable to the original [22].

Discussion

In Czech neuropsychology and neurology, PM represents a new and unexploredtopic [19,20]. The purpose of the pres­ent study was to introduce the concept of PM to a broader Czech neuroscience audience, and to make available the first measurement tool for PM assessment (MIST). PM assessment in clinical populations and different age groups requires data on PM performance in healthy pop­ulations on MIST [10]. Therefore created a standardized Czech version (supervised by the original author of MIST) with back translation of the administration, scoring and original‑based record sheet [18]), val­idating the translated version of MIST on healthy Czech subjects.

We found performance on the MIST Sum­mary score (which indicates general PM performance) to depend on certain demographic characteristics (Tab. 3). Unlike previous studies [16,18], neither age nor education correlated with the Summary score on MIST. That correlation was not detected may be due to the relatively homogeneous demographic characteristics of the sample under study and the small sample size. Given a larger sample and a greater range of ages, our results may have been more in agreement with those report previously in US studies [16,18,48], which found a significant relationship between age and gender. Regarding the influence of education, our results in adults with a higher level of education are surprisingly in agreement with a normative study published previously, where no dif­ferences were found between groups of college‑ educated persons (13– 15 years of education) and those with a longer record of education ≥ 16 years [18]. Differences between groups of men and women were significant in the MIST Sum­mary score, with a medium effect size. This finding concurs with previous stud­ies [16,18]. As a result, we also present separate descriptive statistics for men and women for normative values on the MIST Summary score. However, considering group size in individual genders, our results should be interpreted cautiously.

Reliability is the rate up to which scores are devoid of errors of measurement. In the present study, we tested the internal consistency of MIST, using Cronbach’s α and also split‑ half reliability. We showed that the inner consistency of the eight MIST trials was insufficiently low (α = 0.50– 0.56), which is consistent with the α values in the normative study [18]. However, comparing the internal consistency of six subscales (delay of 2 versus 15 minutes, type of verbal versus action response, time‑based versus event‑based cue; each score ranging from 0 to 8 points), we obtained high internal consistency and split‑ half coefficients (0.88– 0.95) comparable to the original normative study [18].

Validity generally means that the test measures the construct (presently, the construct of PM, compare with Fig. 1) that it is supposed to measure [49]. MIST content validity is shown in Fig. 1. The PM process model shows the basic processes that any test designed to measure PM should adhere to. Fig. 1 shows MIST as a test covering the basic PM processes (intention formation, intention retention, intention initiation, and intention execution relative to different lengths of delay (2 and 15 minutes, 24 hours), cue modal­ity (time‑  and event‑based) and the type of response (verbal and action)). Correlation between the MIST Summary score and all basic MIST subscales may been seen in Tab. 4. The only exception, the subscale of delay after 2 minutes (medium effect size), may be the result of the ceiling effect (see Tab. 3), in agreement with previous studies [16,18]. The construct validity of MIST has been proven in previous studies and surveys [16,18,22,28,50]. In the present study, we show the construct validity of MIST in connection with whether some of the MIST indices correlate with other well established measures of psychic efficiency and effectively delim­­it the construct of PM as measured by MIST in healthy individuals: the MIST Summary score is at the medium level of association with RAVLT delayed recall. This appears to represent a retrospective component of PM (see Fig. 1). Also with regard to TMT‑ B, a test generally viewed as an indicator of mental flexibility, di­vided attention and set shifting [51]. In our opinion, this is a component associated with intention execution, an act requiring considerable mental flexibility, and monitoring (executive processes; see Fig. 1). As follows from functional imaging stud­ies, tasks for PM activate the rostral prefrontal cortex [8,52]. The MIST Summary score correlates to a slightly lower degree with the total score of TOL, which is a task regarded as a paradigm of exec­utive functioning, implicit learning and planning in particular [39], thus lending support to previous findings. It also cor­relates significantly with the Spatial Span forward, a task significantly activating visual focused attention. This is also one of the aspects necessary for effective performance of MIST consisting of visual monitoring and awareness of time while simultaneously solving a parallel task (Word Search Form). Correlation with the Czech National Adult Reading Test (CART), i.e., a test of premorbid intelligence and semantic memory, is another interesting feature. Since the construct of intelligence is a general factor with significant impact on most of the tasks for mental efficiency [53], this correlation is not surprising. The last and lowest medium‑ level correlation is that between MCT and the MIST Summary score attributable to the relationship with memory processes in MCT (a verbal memory test based on an interference paradigm), where resistance to interference is probably one of the basic mechanisms essential for successful performance on MIST (with the particular tasks interfering with one another at the time of execution and in a sequence of event‑based cues). Critically speaking, in all of the above mentioned coefficients of correlation (Tab. 2) their coefficient of determination (as a rate of the joint variability of both variables) was r2 = 0.14– 0.18, which is about one tenth to one fifth of their joint variance. Hence, it cannot be argued that any of the mentioned vari­ables would share a higher proportion of joint variance with MIST.

The present validation study has some limitations. The size of the cohort and its members’ average education and age (a relatively homogeneous group), both clearly restrict any generalization of the results to the Czech population as a whole, especially in individuals with a lower level of education or age. The results of reliability analyses may have differed consid­erably if we also had use of data from clinical groups (e. g., from Parkinson’s dis­ease with dementia). This would certainly lead to greater variability in the scores obtained and in the number of errors [54].

The main objective of the present study was to make a standardized tool for PM measurement available and to adapt the MIST Czech version to clinical practice. MIST enables analysis of the basic components and processes of PM (intention formation and retention (at different lengths of delay), intention retrieval and execution, recognition, time‑  and event‑based PM differentiation and error analysis). PM is a significant component of our mental life, and its measurement may prove to be of major clinical consequence for the dia­gnosis and treatment of patients with memory disorders. We hope this tool will open the door in the Czech Republic to further research on PM; a construct close to our everyday functioning and potentially important and sensitive in neurodegenerative diseases causing memory disorders.

The authors declare they have no potential conflicts of interest concerning drugs, products, or services used in the study.

The Editorial Board declares that the manu­script met the ICMJE “uniform requirements” for biomedical papers.

Mgr. Ondrej Bezdicek

Department of Neurology and Centre of Clinical Neuroscience

Charles University, 1st Medical Faculty and General University Hospital in Prague

Katerinska 30

128 21 Prague 2

e-mail: ondrej.bezdicek@gmail.com

Accepted for review: 17. 10. 2013

Accepted for print: 23. 1. 2014

Přílohy

1. Instrukce Testu paměti pro záměry (MIST)
2. Test na prospektivní paměť (MIST) a jeho administrace