Systematic Review and Meta-analysis of Cognitive Interventions for Children With Central Nervous System Disorders and Neurodevelopmental Disorders

Kristen E. Robinson, PhD; Eloise Kaizar, PhD; Cathy Catroppa, PhD; Celia Godfrey, DPsych; Keith Owen Yeates, PhD

Disclosures

J Pediatr Psychol. 2014;39(8):846-865. 

In This Article

Discussion

The results of the systematic review can be viewed as a glass half empty or half full. On one hand, the meta-analyses provide evidence of the benefits of cognitive interventions, with significant and positive mean effects in all outcome domains aside from inhibitory control. They were large in magnitude (i.e., >0.75) for attention, working memory, and memory tasks, and small in magnitude (i.e., <0.3) for academic achievement and for behavior rating scales assessing attention and working memory. This finding is consistent with the results of other meta-analyses, suggesting that cognitive interventions promote greater change in the cognitive skills they target than in broader measures of function (Melby-Lervåg & Hulme, 2013). Nevertheless, significant improvement was demonstrated in all domains. This is largely consistent with the conclusions of other previous reviews.

On the other hand, the overall quality of evidence was judged to be very low in all outcome domains based on the GRADE ratings, so that we are uncertain about the estimates of treatment efficacy. As is often the case for behavioral interventions, most studies suffered from some risk of bias, usually because of the lack of blinding of participants or study personnel. Another significant issue was the inconsistency of outcome measures, which we believe greatly contributed to the substantial heterogeneity in results within outcome domains. The lack of standards or guidelines in this regard has been problematic. The limited number of studies and participants further reduced the precision of effect size estimates. Publication bias was likewise substantial, leading to major reductions in the estimated effects of intervention on several outcomes. All in all, the limited quality of the evidence provides little confidence in the robustness of the meta-analytic results.

Because of the small number of studies and total participants within any given outcome domain, we were severely limited in our ability to conduct the moderator analyses originally planned. However, we did find evidence for larger effects in children with neurological disorders or acquired brain injuries than in children with neurodevelopmental disorders, at least for attention tasks and working memory tasks, which were the only two outcome domains that included at least two studies of each type of disorder. The reasons for larger effects in children with neurological disorders and brain injuries are not clear. Moreover, given the limited number of studies involved and the limited quality of the evidence overall, this finding must be considered cautiously, and certainly warrants replication.

Thus, the results of the systematic review are far from definitive. They provide some evidence of a positive benefit from cognitive interventions, especially for children with neurological disorders and acquired brain injuries, but the findings cannot be regarded as robust or reliable based on the overall very low quality of the evidence. Thus, the findings do not form the basis for strong recommendations regarding the use of cognitive interventions in clinical practice, although they suggest sufficient potential benefit to warrant further study. In this regard, the review highlights the need for additional clinical trials that evaluate the efficacy, as well as effectiveness, of cognitive interventions. Controlled trials comparing different interventions to one another and to appropriate controls may be especially informative.

To add appreciably to the existing literature, future studies need to be methodologically rigorous, preferably meeting the Consolidated Standards of Reporting Trials (CONSORT) guidelines for clinical trials (Schulz, Altman, Moher, & CONSORT Group, 2010). They should be designed to measure longer-term effects, rather than only immediate posttreatment outcomes, to determine whether any benefits of the interventions are sustained. Further, they should strive to use widely accepted outcome measures to enable cross-study comparisons, such as those recommended by the NIH Common Data Elements Project (e.g., McCauley et al., 2012). Lastly, to facilitate future systematic reviews, publications based on future trials should provide complete statistical data for all outcome measures, ideally to include standardized differences and their standard errors, as well as the correlations among effects when more than one outcome is assessed.

We acknowledge that conducting rigorous clinical trials is a significant challenge, and that the exigencies of journal word limits and editorial constraints sometimes prevent the full publication of all study details and results. Moreover, the current fiscal environment makes it difficult for investigators to obtain funding to support the development of promising interventions or to conduct large-scale clinical trials evaluating the efficacy of existing interventions. Only about half of the studies included in the review reported support through external or internal funding. Nevertheless, cognitive interventions are typically fairly intensive, and require substantial investment in time and energy both by children and their parents. Moreover, some cognitive interventions are being commercially marketed to families and children, at substantial cost to them not only in time but also financially. This raises concerns about the costs versus benefits of cognitive interventions, especially given their uncertain efficacy. Thus, we believe additional rigorous research is imperative.

processing....