A second analysis of data from a previous study using electroencephalography (EEG) to detect residual cognitive function and conscious awareness in patients otherwise appearing to be in a vegetative state now calls those findings into question, researchers say.
The previous study, published in The Lancet last November and reported by Medscape Medical News at that time, suggested that responses on EEG showed residual awareness in 3 of 16, or almost 20%, patients otherwise meeting the clinical definition of a vegetative state.
"Despite rigorous clinical assessment, many patients in the vegetative state are misdiagnosed," the authors, led by Damian Cruse, PhD, from the Center for Brain and Mind at the University of Western Ontario in London, Canada, concluded at that time. This new EEG method, they said, "could allow the widespread use of this bedside technique for the rediagnosis of patients who behaviourally seem to be entirely vegetative, but who might have residual cognitive function and conscious awareness."
This new analysis, published online January 24 in The Lancet, was carried out by an outside group that had set out to reanalyze the data supplied by Dr. Cruse and colleagues with a view to corroborating them.
"We set out to validate the original Lancet study findings about EEG, not disprove them, because it is critically important that tests purporting to assert the presence of consciousness be carefully vetted by peer review," said senior author Jonathan Victor, the Fred Plum Professor of Neurology and professor of neuroscience in the Feil Family Brain and Mind Research Institute at Weill Cornell Medical College and a neurologist at New York-Presbyterian/Weill Cornell.
Instead, they conclude that the statistical analysis used in the paper by Cruse and colleagues was based on faulty assumptions, Dr. Victor told Medscape Medical News.
"When we used a second statistical method, we still found evidence for the responses in the normal subjects, which we expected because the assumptions of their method were sound for that population, but for the patient population, we did not find anything above the level of chance," he said.
Dr. Jonathan Victor
Still, Dr. Victor points out, "we're not concluding that the patients were unconscious, we're concluding that their method found no evidence for consciousness."
Detecting Covert Awareness
Previously, Dr. Cruse and colleagues, with senior author on the paper, Adrian M. Owen, PhD, formerly from the Medical Research Council Cognition and Brain Sciences Unit in Cambridge, United Kingdom, in collaboration with Steven Laureys, MD, PhD, and colleagues from the Comascience Group, Cyclotron Research Center, and Neurology Department at University Hospital of Liège, Beligium, reported they had found several cases of patients in an apparently vegetative state who nevertheless showed signs of brain activation using functional MRI (fMRI).
The scientists were able to show reliable differentiation of response on fMRI to questions that could be answered "yes" or "no" by asking the patients to imagine themselves playing tennis or walking through their home — tasks the researchers knew would engage different parts of the brain.
However, use of fMRI in these patients is challenging, they noted. In addition to the cost and availability of scanners, there is the physical stress of transferring the patients to an fMRI facility. Further, involuntary movements and the presence of the metal plates and pins often used in patients with traumatic injury "can completely rule out fMRI," the authors pointed out.
Dr. Adrian M. Owen
In their study, they instead investigated the use of EEG for this purpose. It answers many of these issues and can be performed at the bedside.
In the EEG record, the authors explained, motor imagery is shown by reductions in power, called event-related desynchronizations, on the μ or β frequency bands (7 - 13 Hz and 13 - 30 Hz, respectively), over appropriate areas of the motor cortex: for hand movements, the lateral premotor cortex, and for toe movements, the medial premotor cortex.
In some individuals, these event-related desynchronizations were accompanied by increases in power, called event-related synchronizations, over motor areas that are collateral to or surrounding the desynchronization.
"With classification techniques, the form of motor imagery being done by a conscious individual can be accurately identified on the basis of the EEG responses alone," Dr. Cruse and colleagues wrote. "We investigated whether these general principles could be adapted to reliably detect covert conscious awareness in a convenience sample of patients who seemed entirely vegetative on the basis of repeated and thorough clinical assessment by specialist teams."
The study included 16 patients with traumatic (5 patients) and nontraumatic (11 patients) brain injury who met the Coma Recovery Scale–Revised definition of vegetative state, as well as 12 healthy control participants. All participants were repeatedly asked to imagine the movements of making a fist and wiggling their toes on command, and their responses were analyzed for "robust evidence of appropriate, consistent, and statistically reliable markers of motor imagery, similar to those noted in healthy, conscious controls."
They found that 3 (19%) of the 16 patients were able to produce these responses, for a classification accuracy of 61% to 78%.
Interestingly, however, 3 of the 12 healthy control patients could not produce reliable responses to the commands given. The researchers point out that even with feedback training, some individuals cannot provide responses that would allow for reliable classification, "so-called brain–computer interface illiterates," they note.
"The absence of a positive EEG outcome for 3 (aware) controls emphasizes the importance of interpretation of only positive results in patients, because this finding shows unequivocally that a null EEG outcome does not necessarily indicate an absence of awareness," they caution. Along with behavioral assessment and functional neuroimaging, many testing sessions with this EEG technique over the course of days will allow patients an increased opportunity to demonstrate any covert awareness, if it exists, they added.
Statistical Assumptions
In the current analysis, the authors, with first author Andrew M. Goldfine, MD, from the Burke Medical Research Institute and Weill Cornell Medical College, point out that this previous finding by Cruse and colleagues, if confirmed, "has major implications for diagnosis and care of severely brain-injured patients."
Both groups are part of the same James S. McDonnell Foundation–funded consortium, with the similar aim of developing methods of determining level of function among patients with severe brain injury, Dr. Victor said. However, they were concerned about the validity of the methods used in the paper by Cruse and colleagues, "because of the difficulty of the task, and its critical reliance on certain statistical assumptions," the authors write. "To allow us to test the validity of the method, Cruse and colleagues graciously supplied their data and analysis software."
Basically, they conclude that the method used in that study "is not valid because the patient data do not meet the assumptions of their statistical model."
For example, the model didn't allow for correlations between trials that were conducted within a few seconds of each other and in a block, Dr. Victor explained. "So they weren't effectively independent rolls of the dice. The problem is that the statistical model used in the original report ignored these dependencies, and analyzed data as if trials within the same block were independent rolls of the dice."
Correlations between nearby trials and blocks are likely induced by fluctuating artifact and arousal states among these patients, the authors note. When these factors are taken into account, they write, "there is no statistical evidence for task performance in patients." The model "generally suffices" among the healthy persons, where such artifact contamination is minimal, they note. These findings cast doubt not only on this paper but also another paper from this group published last year looking at the relationship between etiology and covert cognition in patients in a minimally conscious state, they write.
In another analysis, to account for dependence among trials, they determined significance using a permutation test that recognized the block design, they write. "With this approach, the positive normals remained significant, but only one patient (P13) remained significant (p = .0286, lowest possible p value with 4 blocks)," they write. When they corrected for multiple comparisons, the positive normals remained significant but none of the patients did.
The method used by Cruse and colleagues, a multivariate method called support vector machine, or SVM, and other such methods are particularly useful in EEG analysis for brain–computer interface applications, they write, where "subjects can confirm task performance and the consequences of classifier failure are limited to reduced device performance. But in the diagnostic setting (e.g., determination of consciousness, genomic diagnosis of cancer), classifier failure can misinform clinical decision making, with major consequences for patients and families."
"Given this, and the ease of dissemination of EEG technology, standards of demonstration of validity need to be high. Our analysis suggests that the approach of Cruse et al. falls short of this standard," they conclude.
The Authors' Reply
In the same publication, the authors, led by Dr. Cruse, respond in detail to the findings of the reanalysis. Goldfine and colleagues, they write, "make some interesting points about the choice of statistical model when seeking to identify covert command-following in severely brain-injured patients. Their unconventional cross-validation approach does suggest that the EEG responses of two of our three positive patients became less consistent across time, and argues for future iterations of the task structure to be altered to accommodate this."
Notwithstanding the methodologic concerns of Goldfine and colleagues, they note, "their reanalysis only pushes two of our three positive patients to just beyond the widely accepted p < 0.05 threshold for significance—ie, to p = 0.06 and p =0.09, respectively. To dismiss the third patient, whose data remain significant, they state that the statistical threshold for accepting command-following should be adjusted to account for the number of patients who have been assessed (a so-called multiple comparisons correction). We know of no groups in this field who routinely use such a conservative correction with patient data, including Goldfine and colleagues."
In this case, they note, the only reason to do so would be if they had no a priori hypothesis. However, they point out that in their original publication, after reviewing previous papers, they concluded that the findings confirmed that a population exists who meets all the behavioral criteria for vegetative state but retains a level of covert awareness that can't be detected through behavioral assessment. "Our a-priori hypothesis could hardly have been clearer," Dr. Cruse and colleagues assert.
They are also reassured by the fact that in 2 of the 3 positive patients, fMRI performed the same week the EEG was done confirmed awareness in these patients, they point out.
Their goal, they conclude, "like that of Goldfine and colleagues, is to develop increasingly sensitive tools to identify covert command-following and, in that spirit, we have recently published a method that more formally addresses many of their current concerns," they conclude. "Clearly, it is only through the continuing improvement of our complementary approaches that we will converge on the optimum methods for accurately identifying covert awareness, where it exists, in every severely brain-injured patient."
Rebuttal
Dr. Victor and colleagues have published a reply to the authors' reply on the Cornell site here.
In it, the Cornell group defends their reanalysis of the original study by Cruse and colleagues but conclude with the observation that Cruse et al. have now adopted a different approach, "that appears to address all of the statistical concerns, "including the trial design (randomized, not blocked), the statistical analysis (permutation test no binomial), and the classifier (simple Bayes, not Support Vector Machine," Dr. Victor notes.
The authors have disclosed no relevant financial relationships.
Medscape Medical News © 2013 WebMD, LLC
Send comments and news tips to news@medscape.net.
Cite this: Questions Raised on EEG to Detect Awareness - Medscape - Jan 30, 2013.
