The Pediatric Risk of Mortality Score: Update 2015

Murray M. Pollack, MD; Richard Holubkov, PhD; Tomohiko Funai, MS; J. Michael Dean, MD; John T. Berger, MD; David L. Wessel, MD; Kathleen Meert, MD; Robert A. Berg, MD; Christopher J. L. Newth, MD, FRCPC; Rick E. Harrison, MD; Joseph Carcillo, MD; Heidi Dalton, MD; Thomas Shanley, MD; Tammara L. Jenkins, MSN, RN; Robert Tamburro, MD, MSc

Disclosures

Pediatr Crit Care Med. 2016;17(1):2-9. 

In This Article

Discussion

This revised dichotomous outcome prediction model for PRISM (PRISM IV) functioned very well with an excellent calibration and discrimination that was equivalent to the performance of the original PRISM III 12-hour model despite changes that could have impacted performance. First, using only the first PICU admission during hospitalization, combined with modeling, the outcome at hospital discharge instead of PICU discharge, was important for quality assessment because PICU discharge decision making is an important aspect of PICU quality. For example, a prematurely or inappropriately discharged PICU patient with a subsequent PICU readmission during the same hospitalization was previously credited as a good outcome for the first admission, whereas the subsequent admission had an additional mortality risk credited to their subsequent PICU admission. Therefore, the subsequent PICU admission risk was inflated although it was associated with the premature or inappropriate discharge. Previously, it had not been possible to develop a well-performing predictor using only the first PICU admission and hospital outcome, but this hurdle has been overcome to the overall benefit of model credibility. Second, changing the sampling period from the first 12 hours of care to a significantly shorter time period (2 hr before admission to 4 hr after admission for laboratory data and the first 4 hr of PICU care for other physiologic variables) better separates the PRISM score from therapies but could have resulted in an inadequate sample of physiologic data. This modification was also important because the routine of repeating preadmission laboratory data upon PICU admission, common when PRISM was initially developed, has changed in most institutions. The PRISM physiologic variables and their ranges[5] did not change, only the sampling period changed.

One other change to the PRISM model was required by a significant practice change. Admission of cardiovascular patients for "optimizing" therapy or observation before their intervention is now common in many institutions, and this necessitated a new definition of the PRISM observation period. The decision algorithms to determine the appropriate observation period were created to minimize the potential for "gaming" the observation period. These decision algorithms worked very well with excellent performance within the cardiac and the medical surgical PICUs and within the subsets of cardiac and noncardiac intervention patients. Finally, when PRISM was initially developed, the scores for physiologic derangements for each variable were calibrated to mortality odds ratios; so, the PRISM score for each physiologic variable range represented proportional risk. Over time, new therapies have evolved, and these equivalencies could have changed. We were able to test and adjust this by partitioning PRISM into its five major subcategories. The final predictor partitions the PRISM physiologic variables into the neurologic and nonneurologic components for outcome prediction.

This PRISM IV prediction algorithm based on the first 4 hours of PICU care as a predictor of survival versus death performed as well as the earlier PRISM III 12-hour prediction model, although the changes had the potential to reduce the performance. This performance is predicated on the importance of the physiologic status as the core of the conceptual framework for outcome prediction in the PICU. Recently, we demonstrated that using this core framework, we were able to extend prediction to functional status outcome and mortality. Methods dependent on categorical variables, including those using discharge diagnostic classifications, may not have similar potential to predict functional status as an outcome because they lack the conceptual framework central to pediatric intensive care, treating and maintaining physiologic stability.

The development and maintenance of PRISM and its algorithms are based on the conceptual approach that physiologic dysfunction is the core principle underlying severity of illness and can be assessed independent of computing morbidity and mortality risks. This analysis focuses on the critical details around the current data collection practices. Other categorical factors, such as age, diagnoses, or postintervention status, modify the relationship between physiologic status and risk and enable accurate and reliable estimates of mortality and morbidity risks. In order to maximize the utility of PRISM, we have not included therapies, such as mechanical ventilation for outcome prediction, because PRISM algorithms when used for quality assessment uses physiologic profiles to assess the effectiveness of therapy—conflating physiologic status with therapies would detract from the reliability of this assessment. Similarly, we have not used socioeconomic variables to enable insights into these factors after adjusting for patient status. The potential benefit of this approach is evident from the statistical significance of the SMRs for insurance status in this analysis; others have found associations of socioeconomic factors with severity of illness.[11,12]

The reference sample for this PRISM IV predictor is the PICUs in the second funding cycle of the CPCCRN. PICU quality studies using the published algorithm in this report will be able to compare themselves with the CPCCRN units (external benchmarking) and follow their own performance over time (internal benchmarking). There are advantages and disadvantages to any reference sample. A significant advantage of this sample is that the units have relatively uniform characteristics; they are all large research-oriented units in free standing or "hospital within a hospital" teaching institutions. The characteristics and the patient populations of the individual sites are clearly detailed.[6] The data have been prospectively collected by dedicated staff with the rigor of National Institutes of Health–supported research and with the oversight of a data coordinating center; the data are contemporaneous, and the sample size is sufficiently large for all statistical analyses. Other reference groups may be substantially larger, including the original PRISM III sample and, more recently, the Pediatric Index of Mortality (PIM) 3 sample.[5,13] The later, in particular, is a very large sample of PICUs in the United Kingdom, Australia, Ireland, and New Zealand, but the organizational characteristics of the PIM3 PICUs have not been detailed, and presumably, there are other differences because of the regional diversity. In addition, although a very larger sample size offers statistical advantages, issues of data reliability may become important, especially if the data have been routinely collected for quality assessment or other nonresearch purposes instead of for research uses. Prediction algorithms, such as those in this article, will hopefully extend the implications of individual PICU and regional care assessments beyond the reference sample. Units using these PRISM IV algorithms may perform the same, better, or worse than this reference group, and the clear description and uniformity of the reference sample will help these sites understand their results.

Recently, we advocated for the use of a predictor that assesses survival with significant new functional morbidity, intact survival, and death for assessments of PICU care.[6] As part of that effort, we developed and validated the Functional Status Scale (FSS) score, an age-independent method of measuring functional status suitable for large-scale studies; we used the FSS to assess the new morbidity rate in pediatric critical care and developed a prediction model for the simultaneous prediction of both morbidity and mortality.[6,9,14] However, we realize that there will need to be a period of further discussion and use, as well as routine measurement of the FSS score before its acceptance. This analysis and the placement of this PRISM IV prediction algorithm in the public domain do not alter this recommendation. Hopefully, this contribution will be useful while the field of pediatric critical care considers the value of an outcome predictor of three outcomes: significant new functional morbidity, intact survival, and death.

processing....