Systematic Review and Meta-analysis of Cognitive Interventions for Children With Central Nervous System Disorders and Neurodevelopmental Disorders

Kristen E. Robinson, PhD; Eloise Kaizar, PhD; Cathy Catroppa, PhD; Celia Godfrey, DPsych; Keith Owen Yeates, PhD

Disclosures

J Pediatr Psychol. 2014;39(8):846-865. 

In This Article

Results

Summary of Included Studies

The 13 included studies, published from 2003 to 2012, included 643 children in trials examining the efficacy of cognitive interventions. Three studies involved children with central nervous system disorders, most often acquired brain injuries or brain tumors; the other nine studies involved children with neurodevelopmental disorders, with seven of those focusing exclusively on ADHD. Most of the interventions involved computerized training, often CogMed. The interventions were not typically delivered by psychologists, but instead were supervised by parents in children's homes. The number of sessions ranged from 12 to 102 (median = 25), with sessions ranging in length from 20 min to 2–3 hr (median = 45 min). The duration of the interventions ranged from 25 days to 20 weeks (median = 6 weeks). Treatment fidelity was reportedly assessed in nine studies, most often through reviews of computerized session data or session videotapes; however, four studies provided no information regarding treatment fidelity checks.

Risk of Bias

Figure 2 provides a summary of the risk of bias ratings across the 13 included studies; ratings for each individual study and their justification are available in Supplementary Figure 1 https://jpepsy.oxfordjournals.org/content/39/8/846/suppl/DC1. The most significant source of bias was blinding of participants and study personnel, where 50% of the studies were judged to have a high risk of bias. Blinding of study participants and study personnel can be difficult in behavioral interventions, and so this finding is not surprising. A high risk of bias in the blinding of outcome assessment also was fairly common, occurring in 27% of the studies. The risks of bias in allocation concealment (i.e., where investigators could foresee assignment to conditions) and in reporting of outcome data were considered unclear in many studies, largely because of incomplete reporting in the published studies.

Figure 2.

Risk of bias summary.

Effects of Intervention

Figures 3–9 provide forest plots representing the results of the meta-analysis within each outcome domain. We considered overall mean effects of 0.20 to be small, 0.50 to be medium, and 0.80 to be large (Cohen, 1988).

Figure 3.

Forest plot of effect size estimates for attention tasks.

Figure 4.

Forest plot of effect size estimates for working memory tasks.

Figure 5.

Forest plot of effect size estimates for memory tasks.

Figure 6.

Forest plot of effect size estimates for inhibitory control tasks.

Figure 7.

Forest plot of effect size estimates for attention behavior rating scales.

Figure 8.

Forest plot of effect size estimates for working memory behavior rating scales.

Figure 9.

Forest plot of effect size estimates for academic achievement tests.

Attention Tasks. Four studies, which used 14 different measurements, examined the effects of cognitive interventions on attention tasks. Both the overall mean effect (0.860, 95% confidence interval [CI]: 0.410, 1.309) and the hierarchical mean effect (0.320, 95% CI: 0.241, 0.402) were positive and significant (i.e., 95% CIs did not include 0), with the former being large in magnitude and the latter being small to medium. The hierarchical mean effect was substantially reduced relative to the overall mean effect for attention tasks (i.e., to a small-to-medium effect) because one study (Van't Hooft et al., 2007) provided the majority of the positive effect estimates.

Working Memory Tasks. Intervention effects on working memory tasks were examined in seven studies that used 19 different measurements. Both the overall mean effect (0.756, 95% CI: 0.498, 1.104) and the hierarchical mean effect (0.799, 95% CI: 0.712, 0.885) were positive, significant, and large in magnitude.

Memory Tasks. Only two studies examined intervention effects on memory tasks, using 11 different measurements. Both the overall mean effect (0.953, 95% CI: 0.619, 1.287) and the hierarchical mean effect (1.235, 95% CI: 1.035, 1.436) were positive, significant, and large in magnitude.

Inhibitory Control Tasks. Three studies, which used three different measurements, examined intervention effects on inhibitory control tasks. Neither the overall mean effect (0.594, 95% CI: −0.025, 1.212) nor the hierarchical mean effect (0.541, 95% CI: −0.032, 1.114) was significant, although they were both positive and medium in magnitude.

Attention Behavior Rating Scales. Eight studies examined intervention effects on attention behavior rating scales, using 26 different measurements. Both the overall mean effect (0.296, 95% CI: 0.205, 0.387) and the hierarchical mean effect (0.321, 95% CI: 0.293, 0.350) were positive, significant, and small to medium in magnitude.

Working Memory Behavior Rating Scales. Intervention effects on working memory behavior rating scales were examined in three studies that used four different measurements. Both the overall mean effect (0.227, 95% CI: 0.054, 0.401) and the hierarchical mean effect (0.221, 95% CI: 0.194, 0.248) were positive, significant, and small in magnitude.

Academic Achievement Tests. Four studies, using 14 different measurements, examined intervention effects on academic achievement tests. Both the overall mean effect (0.145, 95% CI: 0.066, 0.223) and the hierarchical mean effect (0.353, 95% CI: 0.314, 0.391) were positive and significant; the former was small in magnitude and the latter was small to medium.

Participant Group Differences

Analyses comparing the effects of interventions for children with neurological disorders or acquired brain injuries versus those with neurodevelopmental disorders revealed significantly larger average effect sizes for the former group. For attention tasks, the average effect size for neurological disorders and acquired brain injuries was 0.314 (F = 7.50, p = .006) larger than for neurodevelopmental disorders. For working memory tasks, the average effect size for acquired brain injuries was 1.050 (F = 99.90, p < .001) larger than for ADHD.

Heterogeneity of Effects

Table III summarizes the heterogeneity of effects within outcome domains. According to the standards suggested in the Cochrane Handbook (Higgins & Green, 2011), the results within all domains exhibit substantial (i.e., I2 between 50 and 90%) or considerable (i.e., I2 greater than 75%) heterogeneity. The only exception was for behavior rating scales for working memory. This exception is not surprising, as the standard DerSimonian and Laird (1986) method underestimates the between-study variance to be zero and thus forces I2 to be also zero. In general, the heterogeneity results confirm our decision to use a random-effects meta-analysis approach.

Publication Bias

Funnel plots provided evidence of significant potential publication bias in many outcome domains. After adding potentially missing studies using trim-and-fill methods, the estimated effect size was reduced substantially for attention tasks, working memory tasks, and academic achievement. Publication bias was not substantial for memory or inhibitory control tasks or for behavior rating scales measuring attention or working memory.

Quality of Evidence

Table IV provides a summary of the findings within each domain. In all cases, the quality of evidence was judged to be very low based on the GRADE criteria. Thus, we are uncertain about the effect size estimates in all domains. Study limitations were considered serious in all outcome domains based on the risk of bias judgments. Results were judged inconsistent in all domains other than behavior rating scales for working memory based on the heterogeneity of effects. Results were judged indirect for behavior rating scales, which do not directly assess cognitive or academic abilities. Imprecision was considered serious for studies that examined the domains of memory, inhibitory control, and behavior ratings of working memory, all of which had three or fewer studies and <200 total participants. Publication bias was considered serious or very serious for studies of attention tasks, working memory tasks, and academic achievement. Studies of attention tasks and working memory tasks were credited for large effect sizes.

processing....