Abstract and Introduction
Abstract
Cognitive screening tests such as the Mini-Mental State Examination are widely used in clinical routine to predict cognitive impairment. The raw test scores are often corrected for age and education, although documented poorer discrimination performance of corrected scores has challenged this practice. Nonetheless, test correction persists, perhaps due to the seemingly counterintuitive nature of the underlying problem. We used a causal framework to inform the long-standing debate from a more intuitive angle. We illustrate and quantify the consequences of applying the age-education correction of cognitive tests on discrimination performance. In an effort to bridge theory and practical implementation, we computed differences in discrimination performance under plausible causal scenarios using Open Access Series of Imaging Studies (OASIS)-1 data. We show that when age and education are causal risk factors for cognitive impairment and independently also affect the test score, correcting test scores for age and education removes meaningful information, thereby diminishing discrimination performance.
Introduction
Cognitive screening tests are tools used to screen for dementia and cognitive impairment.[1,2] These tests are validated by assessing their ability to predict clinical diagnoses, such as dementia and mild cognitive impairment, characterized by a decline in cognitive performance.[1–6]
An individual's raw test score is the numerical result of the test, such as the number of correct answers or errors, time to complete the test, or performance rating.[5,7] On their own, these raw scores are generally considered to have no inherent meaning and only become interpretable when compared with preexisting norms or standards from individuals with similar demographic characteristics (e.g., age, sex, ethnicity, and/or educational level).[5,7] This comparison with the corresponding demographic-specific norm is conducted by transforming the raw score into a corrected score. In psychology, this practice is referred to as standardization, adjustment, or correction.
The correction of raw scores is often employed in cognitive screening,[1,8–11] despite having repeatedly encountered sharp criticism pertaining to its poor discrimination performance. First, theoretical concerns were raised together with initial empirical observations,[12–19] and additional support was found in simulation studies.[13]
However, the aforementioned criticism did not noticeably shift the paradigm. One reason might be the lack of an explicit causal framework, which, in our opinion, is necessary for an intuitive, complete understanding of this problem. Therefore, we aim to take a causal approach to address this long-standing debate regarding prediction performance. We provide an explanation for the counterintuitive results in the literature surrounding the problem of the age-education correction in cognitive screening. Specifically, we detail the explicit relationship between the correction of these tests and discrimination performance operationalized as area under the receiver operating characteristic curve (AUC).
Am J Epidemiol. 2023;192(1):93-101. © 2023 Oxford University Press
