Evidence of pHWF Influencing Allergic Disease Risk
Early Studies: Small & Positive
The initial studies evaluating pHWF for allergic disease prevention were relatively small studies (<100 participants) of high-risk infants, randomized to receive pHWF or conventional cows' milk formula (CMF) from birth.[42–44] These studies showed positive results, although evidence was subsequently found to indicate that the results of one of these studies was likely to have been fabricated and has been removed from systematic reviews on this topic.
Some of the positive results from these studies were based on outcomes that were potentially inherently biased towards showing a positive effect of pHWF. The outcome of cows' milk allergy is often reported in these studies, but not all children are given the same opportunity to be diagnosed with this condition. Instead of all children receiving a cows' milk challenge at a specific age, children were only investigated for possible milk allergy in these studies if they exhibited possible symptoms. Take for example, two children enrolled in a study at their birth to investigate the effect of infant formula on development of allergic disease, both of whom develop a cows' milk allergy at 2 months of age during exclusive breastfeeding. One child is assigned to receive pHWF while the other is assigned to CMF at the complete or partial cessation of breastfeeding. The child given the CMF will develop a clinical reaction, which will be diagnosed. By contrast, only approximately 30% of children with a confirmed cows' milk allergy will react to the pHWF, and fewer to extensively hydrolyzed formula. Given the natural history of cows' milk allergy to resolve quite rapidly in most cases, the child given pHWF may have outgrown their reaction by the time the intervention has been completed, and they are introduced to whole cows' milk protein. This has potentially led to underdetection of cows' milk allergy in the hydrolyzed formula groups, and thus overestimating the protective effect of these interventions for this outcome.
The Difference Between 'Intention-to-Treat' and 'Per-protocol' Analyses
The main ways in which results from clinical trials are presented are either intention-to-treat (ITT) or per-protocol (PP). It is important to understand the conceptual differences in these approaches, as the results of these analyses have different strengths and limitations, and can lead to quite different conclusions. Analysis by the ITT principle has become the favored form of analysis for clinical trials. This ensures that the randomization process is the only factor that influences the composition of the groups, and avoids confounding bias. However, it is well known that ITT analysis is a conservative form of analysis, particularly when a substantial number of participants do not receive the allocated intervention, or receive an intervention that they were not meant to. In the case of the studies in this area, many children are breastfed for a substantial period of time, and may never actually be exposed to the allocated formula (nonexposure), while other children are given nonallocated formula during the intervention period (contamination). These effects of contamination and nonexposure will result in a dilution of the observable effect of the intervention if an ITT analysis is used.
By contrast, PP analyses only include those participants who have received the intervention in the way that it was intended. The aim of a PP is to estimate the true underlying, or 'biological', effect of the intervention. In these analyses, infants may be excluded if they have not been exposed to the allocated formula by a particular age or do not receive a 'substantial' dose of formula. However, there are a number of important trade-offs that occur when moving from an ITT analysis to a PP analysis. Exclusion of nonexposed children in a PP analysis results in reduced sample sizes and thus statistical power. More importantly, PP analyses are not protected by the randomization process and bias may be introduced at various points. In this particular context, infants may be excluded if they 'break from the feeding protocol'. Such exclusions may bias the measured effects, as the reason for the violation of the feeding protocol may actually be due to the child reacting to their assigned formula. The decisions concerning what age exposure must occur by, the number of days of exposure and the total quantity of formula received to be considered an 'adequate dose' are all potential factors that need to be considered when designing a PP analysis. As these are arbitrary decisions, the results may be influenced by signals in the data, thus inflating any apparent effects. For these reasons, PP analyses are best viewed as hypothesis generating, rather than hypothesis testing.
More Recently Published Studies: Larger & Mixed Results
More recent research into the role of pHWF for allergic disease prevention evolved to include larger numbers of infants.[18,50–53] Again, these studies only recruited high-risk infants, but randomization of infants occurred before birth (or soon afterwards). Many of these infants were breastfed and then introduced allocated formula at some point in the first year of life, while others never received the formula at all due to prolonged breastfeeding. This is a key difference between these studies and earlier, smaller studies of the effect of pHWF.
The results of these larger trials comparing pHWF with CMF, when analyzed using the ITT principle, generally showed no or very limited evidence of a protective effect for all allergic disease outcomes. By contrast, the PP analysis shows that pHWF may have some protective effect for eczema in some studies, while this effect was not confirmed in others. Specifically, the GINI group reported a reduced risk of eczema (odds ratio [OR] = 0.56; 95% CI = 0.32–0.99) in the first year of life in a PP analysis that only included the 44.7% of children who were exposed to the allocated formula, and were not exposed to another formula, by 4 months of age. When the MACS cohort was analyzed in the same way, there was no evidence of a protective effect (OR = 1.03; 95% CI = 0.55–1.93, 43.7% of children could be included). Unlike early life eczema, the PP analyses for both MACS and GINI consistently demonstrate no apparent effect of pHWF on the outcomes of childhood asthma (OR for GINI = 1.64; 95% CI = 0.59–4.53 and OR for MACS = 0.73; 95% CI = 0.37–1.45), and allergic rhinitis (OR for GINI = 0.95; 95% CI = 0.55–1.63 and OR for MACS = 0.80; 95% CI = 0.39–1.64). The impact on eczema but not asthma shown in the GINI study has been argued to indicate pHWF does not influence the atopic march.
Various decisions need to be made as to which participants are included in PP analysis, and many of these decisions may influence the study results. As such, it is important that the effects shown in PP analysis can be replicated. Unfortunately, replication of PP results has proven difficult in this area. The lack of a clear ITT effect of pHWF for allergic disease prevention, and the inconsistency of PP results, creates difficulty in interpreting the role of pHWF in the prevention of allergic disease.
The largest studies of the effect of pHWF to date have randomized children prior to birth, and introduced the allocated formula at the cessation, or partial cessation of breastfeeding.[18,51] Although this mimics the 'real world' use of these formulae, it creates a situation where randomized children may receive limited, or even no, exposure to study formula within the 'critical window' of 4–6 months in early life. In the GINI study 62%, and in MACS 52%, of infants received their allocated formula by 4 months of age. As such, any beneficial effect of pHWF is extremely difficult to demonstrate in ITT analysis (as 40–50% of children are not exposed during the key period).
What Do Systematic Reviews Tell Us?
It is important to consider all available information when considering the effectiveness of an intervention. The highest level of evidence to support an intervention comes from multiple, well conducted, randomized trials providing consistent effects. Systematic reviews aim to identify all studies that have investigated the effect of an intervention, and summarize its effectiveness. This can be achieved statistically using meta-analysis, to come up with a summary, or pooled, estimate of effect.
A number of systematic reviews have assessed the evidence that pHWF may help reduce the incidence of allergic disease in infants.[25,55,56] Like original publications, systematic reviews can also have methodological limitations. Specific limitations of the review by Alexander and Cabana include no duplicate assessment of study eligibility, and extraction of data, lack of use of Medical Subject Headings search terms, lack of search for unpublished 'gray' literature, inclusion of studies with high likelihood of bias due to nonrandomization, nonblinding or both, and inclusion of studies with high rates of loss to follow-up, and the review being funded by an organization with a financial interest in the results of the review. The review by Szajewska and Horvath was methodologically stronger in terms of duplicate selection of studies and checking of data, but included quasi-randomized trials, only assessed the effect of a single pHWF, and again was funded by an organization with a financial interest in this area. For these reasons, the strongest review evidence on this topic comes from the Cochrane review on this topic by Osborn and Sinn. Furthermore, the majority of guideline documents use this review as a key reference to support the recommendations on this issue, which is why the authors have focused on the meta-analysis conducted by this group.
The systematic review and meta-analysis conducted within the Cochrane review on this topic estimated that prolonged feeding with pHWF resulted in a 27% (95% CI: 10–41%) reduced incidence of 'any allergic disease' in infancy when compared with feeding with intact CMF. The Cochrane review did not include the recently published results from the MACS trial, and inclusion of the data from the MACS trial probably results in a reduced estimate of the effect of pHWF. It should be noted that the MACS trial was previously omitted from the Cochrane reviewer on this topic, based on an incorrect assumption made concerning an interim report supplied by the organization that initially funded the project, without directly contacting the study investigators to confirm these details. We have informed the authors of the Cochrane review on this topic, and requested that they consider this study when they next update their review.
The authors have recently raised the possibility that publication bias may have impacted on the systematic review of evidence concerning pHWF for allergic disease prevention. Publication bias arises when small studies showing null, or unfavorable, results are less likely to be published than small studies that show strong effects. This can result in overestimating the effect of an intervention when the results of multiple trials are combined. Publication bias occurs for a range of reasons, but is largely due to it being more difficult to publish negative study results. The creation of clinical trial registries has helped address this issue, as it is now compulsory (for future publication) to have clinical trials registered in a publicly accessible format, where the design and outcomes to be assessed are prospectively documented. This means that groups undertaking systematic reviews are able to identify unpublished studies and include the results in the synthesis of evidence. Unfortunately, the majority of the trials conducted on pHWF for allergic disease prevention were initiated and published before 2005, when the trial registries were initiated. As such, it is not possible to be confident that all trials on this topic have been identified.
The importance of assessment of potential publication bias in systematic reviews has been recognized for some time, although not all reviews have formally taken this into consideration. Unfortunately Osborn and Sinn did not assess potential publication bias. When a funnel plot is generated from the published meta-analysis by Osborn and Sinn (Figure 1), there is evidence of asymmetry; that is, there is an absence of smaller studies (denoted by larger estimates of variance) that show null or nonprotective effects (denoted by estimates of effect that are closer or above 1). This pattern of results is suggestive of publication bias. Alexander and Cabana examined potential publication bias, and also reported asymmetry in the funnel plot of studies addressing this topic. Szajewska and Horvath considered publication bias, but elected not to formally test for it based on limited number of studies available.
Funnel plot of the Cochrane meta-analysis 14.1. Prolonged feeding: Partially hydrolyzed whey formula versus cow's milk formula; outcome 1: any allergy . This shows a funnel plot of a meta-analysis from the Cochrane review of this topic . The x-axis represents the SE of the estimate for each study (largely determined by the study size): larger studies have smaller SEs, resulting in more precise estimates of the true effect of the intervention, and are plotted further to the left on the funnel plot. The y-axis represents the estimated effect from each study: a value of 0 indicates no effect (RR of 1) while studies with an effect estimate smaller than 1 indicate that partially hydrolyzed whey formula is protective for the outcome of 'any allergy'. In the absence of publication bias it is expected that the funnel plot will show symmetry around the pooled estimate of effect (a value of −0.32, which corresponds to the odds ratio of 0.73). As can be seen, there is distinct asymmetry in this plot, with an absence of small studies in the area indicated by the circle (Harbord p = 0.06) .RR: Relative risk; SE: Standard error.
Asymmetry in a funnel plot is a possible indicator of publication bias, but this pattern of results may also arise from other causes. Specifically, there may be methodological differences between larger studies and smaller studies that reduce the impact of pHWF. It is plausible that the effect of hydrolyzed formula may be different in children who have been breastfed prior to initiation of formula. Human breast milk contains a host of immunologically active components. These components of breast milk may independently help induce allergen tolerance while preventing sensitization and allergic disease. Feeding CMF following a period of breastfeeding may be less likely to induce allergic disease than feeding CMF from birth. As such, the observable benefit associated with feeding pHWF may be reduced when compared with feeding CMF following breastfeeding. This effect may help in explaining some of the variability in the study results: the smaller studies have only randomized infants who are exclusively formula feed (without prior breastfeeding) while larger studies, which randomize infants prior to birth, compare children who were generally breastfed and then introduced to the study formula at some point in the first year of life, and often after the first 4 months of life.
Limitations of Studies on Hydrolyzed Formula
All studies of pHWF to date have important limitations. As previously noted, it is difficult to successfully blind parents and research staff to the nature of hydrolyzed formula due to differences in taste and smell. This is a greater issue for trials of extensively hydrolyzed formula, where these differences are more marked. This potential for unblinding may lead to bias influencing the study findings. There has also been a lack of assessment of severity of allergic disease – the vast majority of allergic disease in infancy and early childhood is relatively mild disease that causes limited impact on the child and the family. The more severe cases of allergic disease are less common, but induce a much greater burden. It has been suggested that only mild cases of disease are potentially being prevented or delayed. Furthermore, it has been suggested that hydrolyzed formulae do not prevent allergic disease, but simply delay the diagnosis of these symptoms. Arguing against this possibility is the finding from the GINI study that the cumulative incidence of eczema from birth to 6 years was lower in infants feed pHWF (aRR: 0.79; 95% CI: 0.64–0.97 from ITT analysis). However, it should be noted that although prevalence of eczema from 4–6 years was lower in the PP analysis (aRR: 0.53; 95% CI: 0.34–0.84), this finding was not confirmed in the ITT analysis (aRR: 0.83; 95% CI: 0.61–1.12).
Expert Rev Clin Immunol. 2013;9(1):31-41. © 2013