Issues in Experimental Design
The prototypical fMRI experimental design is a "boxcar" design in which two conditions are alternated over the course of a scan in a single subject. This is a categorical, blocked, subtractive design. Categorical because the experiment examines two levels of a category and blocked because, for most hypotheses of interest, these periods of time will not be utterly homogeneous but will consist of a block of several "trials" of some kind presented together. For example, a given block might present a series of faces to be passively perceived, designed to evoke a particular cognitive process, such as face perception. These "experimental" blocks are alternated with "control" blocks, which are designed to evoke all of the cognitive processes present in the experimental block except for the primary cognitive process of interest. The inferential framework of "cognitive subtraction" allows one to attribute differences in neural activity between the two conditions to the cognitive process of interest, which has been putatively isolated by subtraction. Although the application of cognitive subtraction to imaging was a major innovation when originally introduced by Posner and colleagues,[7,44] subtraction was originally conceived by Donders (1868) for studying the chronometric substrates of cognitive processes. Evidence supports the notion that the assumptions required for this method may not hold and thus may lead to erroneous interpretation of imaging data.
Cognitive subtraction relies upon two assumptions: "pure insertion" and linearity. Pure insertion is the idea that a cognitive process can be added to a preexisting set of cognitive processes without affecting them. This assumption is difficult to prove because one would need an independent measure of the preexisting processes in the absence and presence of the new process. If pure insertion fails as an assumption, a difference in neuroimaging signal between the two conditions might be observed not because of the evocation of the cognitive process of interest but because of an interaction between the added component and preexisting components. For example, failure of this assumption in functional neuroimaging can be illustrated with working memory studies that have used delayed response paradigms. These paradigms are composed of a memory maintenance requiring a delay period between a "perceptual" process (in response to the presentation of the item or items to be stored) and a "choice" process (a required decision based upon the item that was stored46). The neural substrates of the memory-maintenance process are proposed to be revealed by a subtraction of the integrated (i.e., averaged, summed, or totaled) functional hemodynamic signal during a no-delay condition (i.e., a block of trials without a delay period) from that during a delay condition (i.e., a block of trials with a delay period). In this example, failure to meet the assumptions of cognitive subtraction will occur if the insertion of a delay period between the "perceptual" and "choice" processes affects these other behavioral processes in the task. That is, these non-memory-maintenance processes may be different in delay trials as compared with no-delay trials.
In functional neuroimaging, an additional requirement must be met for cognitive subtractive methodology to yield nonartifactual results: the transform between the neural signal and the neuroimaging signal must be linear. In several studies, linearity has been found not strictly to hold for the BOLD fMRI system.[36,47] In our example of a delayed-response paradigm, failure will occur if the sum of the transform of neural activity to hemodynamic signal for the "perceptual" and "choice" processes differs when a delay is inserted as compared with when it is not present. In this example, such artifacts of cognitive subtraction might lead to the inference that a brain region displayed delay-correlated increases in neural activity when in actuality it did not -- the differences might, in fact, be associated with changes in other processes. In the following, we present empirical evidence of such a failure.
In addition to requiring the assumptions of cognitive subtraction, blocked designs may be undesirable because, by their nature, they do not allow the randomization of the order of stimuli and are constrained to grouping trials of the same type with each other in time. The consequent predictability of trial type may act as a confound in a blocked experiment. For example, most early imaging studies of the neural substrates of recognition/novelty processing have involved presenting subjects with blocks of either all old (i.e., previously presented during an encoding condition) or all new (i.e., not presented during an encoding condition) stimuli together for judgments of recognition.[48,49] Such a situation highlights the undesirability of being constrained to blocked trial structures. The influence of trial order (blocked or random) on functional neuroimaging data can occur on at least two levels. First, the order of trial presentation may have an effect upon the cognitive processes engaged within the trials themselves. Evidence has been presented for interactions of this kind. Second, blocked or random presentation may affect the cognitive processes during the intertrial interval. Thus, every blocked experiment is confounded by behaviors that may be the result of groups of similar trials being presented together as opposed to the effect of the individual trials.
Despite these concerns, a categorical, blocked experimental design is sometimes desirable. When an experiment is to address a purported cognitive process that (1) is an all-or-none phenomenon (i.e., cannot be subjected to parametric manipulation), (2) is homogeneous in its evocation (e.g., there are not correct or incorrect trials), and (3) cannot be separated by several seconds in time from other cognitive processes (e.g., unlike the delay period of working memory tasks), then other designs offer no advantage. Although cognitive subtraction embraces some highly undesirable assumptions, in some cases, for some cognitive processes, it is the only option available.
A new class of experimental designs, called event-related fMRI, attempts to model signal changes associated with individual trials as opposed to a larger unit of time composed of a block of trials.[51,52] Each individual trial may be composed of one behavioral "event" (such as the presentation of a single word) or several behavioral "events" (such as the presentation of a cue, a delay period, and a motor response in a delayed-response task). In the simplest type of event-related fMRI experiment, the behavioral trials are distant enough in time from one another to allow the hemodynamic response, which results from the hypothesized brief period of evoked neural activity, to run its course fully (e.g., 16 seconds) (Fig. 2). A variety of analysis approaches are available that allow the statistical evaluation of these responses, both with respect to the intertrial interval and with respect to one another. Because the analysis focuses upon individual trials, it is possible to ascribe changes in the neuroimaging signal to the effect of one particular trial type, regardless of when it is presented within the experiment. This feature of event-related designs allows the randomization of stimuli, thus avoiding behavioral confounds that are the result of blocking trials, and allows the separate analysis of functional responses that are identified only in retrospect (i.e., trials on which the subject made a correct or incorrect response).
Schematic diagram of the differences in the design and analysis of blocked versus event-related fMRI designs. In the blocked design (A), nine different trials (black and white boxes) are presented sequentially. The fMRI signal (black line) is evaluated against a boxcar (square wave) reference function that has been smoothed to account for the physiology of the hemodynamic response (gray line). This analysis is sensitive to signal changes between the block of trials (0-16 seconds) versus the block of time without any trials (16-32 seconds). In contrast, in the event-related design (B), one trial occurs every 16 seconds. The fMRI signal (black line) is evaluated against isolated representations of the hemodynamic response (gray solid line and gray dotted line). In this example, different magnitudes of neural responses were produced by the two different trial types. Notably, the event-related design is capable of distinguishing two types of response because of the temporal separation of the individual trials and the use of covariates that model only one trial type (gray solid line) or another (gray dotted line).
In other experiments in which each individual trial is composed of several behavioral "events" (such as in a delayed-response task), event-related fMRI analysis methods allow one to determine separately the neural substrates of temporally dissociable components of behavior. The logic of one implementation of one such design for the study of working memory is as follows38 and is illustrated in Figure 3. If we wish to detect and differentiate the fMRI signal evoked by a series of sequential neural events (i.e., such as the presentation of the stimulus and, seconds later, the execution of the response), one method would be to model the evoked fMRI signal statistically using a set of hemodynamic responses as covariates, each shifted to the appropriate time period where the event of interest is thought to occur. A combination of hemodynamic responses could theoretically be used to model any neural event, even if the event is sustained, such as delay period activity.
The logic for the fMRI data analysis in one such event-related fMRI design is illustrated schematically. Shown are two examples of how the analysis model would respond to different neural responses during trials of a delayed-response paradigm. (A) depicts a scenario in which there is only a brief period of neural activity (first row) associated with both the stimulus presentation and the discrimination/response periods of trials, with no increase above baseline during the bulk of the retention delay. Such neural activity change would lead to a particular profile of fMRI signal change (second row). The model covariates (i.e., hemodynamic response functions shifted to sequential time points of the trial) scaled by their resulting least-squares coefficients are shown in the third row (gray lines, covariates modeling the retention delay; black lines, covariates modeling the stimulus presentation and the discrimination/ response periods). It can be seen that the covariates modeling the retention delay would make only a small contribution to variance explanation in (A). In contrast to (A), (B) depicts a situation in which there is some neural activity increase relative to baseline during the retention delay. In this case, it can be seen that the covariates modeling the retention delay would tend to explain a larger amount of variance in the fMRI signal than in (A). In this way, delay-specific brain activity would be detected by the model.
Analyzed in the manner described, during the performance of a spatial delayed-response task, we observed that several brain regions, including PFC, consistently displayed activity that correlated with the delay period across subjects. This finding suggests that these regions may be involved in temporary maintenance of spatial representations in humans. With this event-related fMRI design, we could be confident that activity observed was not due to differences in other components of the task (i.e., presentation of the cue or motor response) during the behavioral trials. Most important, these results do not rely on the assumptions of cognitive subtraction. An example of the time series of the fMRI signal averaged across trials for a PFC region displaying delay-correlated activity is shown in Figure 4A.
In the same study, we also found direct evidence for the failure of cognitive subtraction (Fig. 4B). We found a region in PFC that did not display sustained activity during the delay (in an event-related analysis) yet showed greater activity in the delay trials as compared with the trials without a delay. In any blocked neuroimaging study that compares delay versus no-delay trials with subtraction, such a region would be detected and probably assumed to be a "memory" region. Thus, this result provides empirical grounds for adopting a healthy doubt regarding the inferences drawn from imaging studies that have relied on cognitive subtraction.
Data derived from the performance of a normal subject on a spatial working memory task. This task comprised both delay trials (dark circles) and trials without a delay period (no-delay trials, diamonds). During no-delay trials, the subject performed a spatial discrimination task that did not depend upon spatial working memory. (A) Trial-averaged fMRI signals from a region in prefrontal cortex that displayed delay-correlated activity (across correct and incorrect trials). The gray bar along the x-axis denotes the 12-second delay (that occurs during delay trials). The delay trials display a level of fMRI signal greater than baseline throughout the period of time corresponding to the retention delay (taking into account the delay and dispersion of the fMRI signal). The peaks seen in the signal correspond to the stimulus presentation and discrimination/response periods. The no-delay trials were brief (about 1 second) and take place at the very beginning of the gray bar only. (B) Trial-averaged fMRI signal from a region in prefrontal cortex that did not display the characteristics of delay-correlated activity. This region displays a significant functional change associated with the no-delay trials and a significant functional change associated with the stimulus presentation and discrimination/response periods of the delay trials but not one associated with the retention delay of delay trials. Because this region displays a greater averaged neuroimaging signal during delay trials than during no-delay trials, it is consistent with failure to satisfy the assumptions of cognitive subtraction.
Semin Neurol. 2000;20(4) © 2000 Thieme Medical Publishers
Cite this: Functional Imaging of Neurocognition - Medscape - Dec 01, 2000.