Planned Home vs Hospital Birth: A Meta-Analysis Gone Wrong

Authors and Disclosures

Author(s)

Carl A. Michal, PhD

Associate Professor, Department of Physics and Astronomy, The University of British Columbia, Vancouver, British Columbia, Canada

Disclosure: Carl A. Michal, PhD, has disclosed no relevant financial relationships.

Patricia A. Janssen, PhD

Associate Professor, School of Population and Public Health, The University of British Columbia, Vancouver, British Columbia, Canada

Disclosure: Patricia A. Janssen, PhD, has disclosed no relevant financial relationships.

Saraswathi Vedam, SciD

Associate Professor & Director, Division of Midwifery, University of British Columbia, Vancouver, British Columbia; Senior Consultant, Division of Research, Midwives Alliance of North America, Washington, DC; Chair, Home Birth Section, Division of Standards and Practice, American College of Nurse-Midwives, Silver Spring, Maryland

Disclosure: Saraswathi Vedam, SciD, has disclosed no relevant financial relationships.

Eileen K. Hutton, PhD

Director, Midwifery Education Program, McMaster University, Hamilton, Ontario, Canada

Disclosure: Eileen K. Hutton, PhD, has disclosed no relevant financial relationships.

Ank de Jonge, PhD

Senior Midwife Researcher, Department of Midwifery Science, EMGO Institute for Health and Care Research, VU University Medical Center, Amsterdam, The Netherlands

Disclosure: Ank de Jonge, PhD, has disclosed no relevant financial relationships.

Appendix. An Analysis of "Meta5.doc," The Computational Tool Used For Random-Effects Meta-Analysis

The random effects calculations in the study by Wax and colleagues made use of a meta-analysis calculator implemented in a spreadsheet that was embedded in a Microsoft® Word document.^{1} The spreadsheet is based upon formulae found in Petitti's meta-analysis text.^{2} The formulae in question are in Table 7-7 on page 102 and on pages 116-117.

Specifically, the errors in the spreadsheet are:

1. In cell W10, which contains Petitti's D value (Δ² in DerSimonian and Laird's notation^{3}), any negative value for D should be replaced with 0. The spreadsheet, however, contained no logic to replace negative values. In cases where D is negative, this could dramatically alter the weightings of the datasets.

2. In the calculation of the adjusted weights (w_i*), which should be given by w_i* = 1/(D+(1/w_i )), the spreadsheet cell reference to the cell containing D was entered not as an absolute cell reference (eg $W$10) but as a relative cell reference (W10), so that it referred to the (possibly incorrect, due to error 1) D value in cell W10, for only the first study. For subsequent studies, the value taken to be D was whatever value happened to be in cells W11, W12, W13, etc. As a consequence of the layout of the spreadsheet, those cells are generally blank, returning values of 0.

3. In the calculation of the CI limits, rather than employing the sum of the adjusted variances (variance_s*), the sum of the raw variances was used. To correct this, the references to cell H10 in cells I10 and J10 should be replaced by 1/sqrt(U14).

4. The spreadsheet provides a negative value for the Q statistic. Because Q is a weighted sum-of-squares, this cannot be correct. Q should have been taken from cell S14.

It is possible that the spreadsheet will be corrected. The original version of the spreadsheet can be found on the Internet archive "Wayback Machine."

After making the described corrections, the spreadsheet appears to implement correctly Petitti's description of the DerSimonian-Laird method. However, the results provided by the spreadsheet still show minor deviations from their expected values. This is the result of a discrepancy between Petitti's algorithm and DerSimonian and Laird's paper. In particular, Petitti makes use of the Mantel-Haenszel variance and OR in calculating the adjusted weights w_i ^*. These do not appear to agree with the variance (s_i ² ) and weighted (natural log of) OR (y_w ) indicated for OR calculations in DerSimonian and Laird's original paper.

As an example of the possible consequences of using the spreadsheet, we consider a random effects model calculation combining the 2 datasets shown below.

The spreadsheet provides a random effects OR of 0.64, with 95% confidence bounds of 0.55-0.73. Simply exchanging the order of the 2 studies changes the result to OR = 2.48 (95% CI, 2.15-2.85).

The correct results from a DerSimonian-Laird random effects calculation^{1} is OR, 1.26 (95% CI, 0.32-4.92). The correct results in this example were calculated using the rmeta package^{4} for the R statistical analysis environment^{5} and verified by hand calculations.

These analyses are shown in the forest plot in the Figure. Clearly, either of the 2 incorrect results (shown in blue and red) would lead to incorrect conclusions -- both results spuriously suggest statistical significance, although the conclusion on the direction of the effect depends on the order in which the studies are entered into the spreadsheet.

Figure. Forest plot showing ORs and CIs for a random-effects meta-analysis of the example data sets. The correct result is represented by the long thin black diamond. The red and blue diamonds represent the 2 possible incorrect results produced by the faulty spreadsheet.

References

1. Basu A. Calculation of summary effects using random effects method. Available at: http://www.pitt.edu/~super1/lecture/lec1171/meta5.doc Accessed January 28, 2011.

2. Petitti DB. Meta-Analysis, Decision Analysis, and Cost-Effectiveness Analysis. 2^nd Ed. Oxford University Press; New York; 2000.

3. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177-188.

4. Lumley, T. rmeta: Meta-analysis. September 29, 2009. Available at: http://cran.r-project.org/web/packages/rmeta/index.html Accessed January 26, 2011.

5. R. The R Project for Statistical Computing. Available at: http://www.r-project.org Accessed January 26, 2011.