Use of Influenza Risk Assessment Tool for Prepandemic Preparedness

Stephen A. Burke; Susan C. Trock


Emerging Infectious Diseases. 2018;24(3):471-477. 

In This Article


Scoring Procedure

The ranking and weighting of risk elements used to answer the 2 standard IRAT risk questions (emergence, impact) was predetermined by the working group of international influenza experts.[10] Discussion and debate about the importance of each risk element to answer the 2 questions resulted in a consensus ranking of risk elements for each question after 2 rounds of ranking all elements. Subsequently, when viruses were evaluated by subject-matter experts (SMEs), their task was to consider a virus solely within the definition of the individual risk element they were scoring. SMEs scored a specifically selected virus of interest; only data related to this strain were considered to avoid confusion over potentially significant strain differences. Each evaluation was conducted in the context of data available at the time of the evaluation. Multiple SMEs scored each risk element, but the maximum number of risk elements scored by any one SME evaluating a virus was set at 3 in an attempt to maintain a high level of expertise, assuming that most SMEs are not experts across all the technical areas represented by the various risk elements. Limiting the number of elements any SME scores also reduced the burden on any one SME, which can potentially shorten the time to evaluate a given virus and removed the possibility of potential bias introduced by an SME scoring most or all elements.

The SMEs provided a point estimate based on a standardized scale of 1–10 using the definition and criteria of the risk element. For each element, the lowest risk score possible is 1, and the highest risk score possible is 10. The SMEs also provided an upper and lower bound of acceptability in scores they would consider reasonable from other experts, assuming the same knowledge base regarding the risk element. In addition, a justification was solicited that summarized the critical basis contributing to the SME's point estimate, as well as his or her judgment of confidence in the quality of the data. Scoring was collected in 2 phases. In the initial phase, preliminary data were reviewed, and variation in the individual risk element scores was noted. A wide range of point estimate scores for a particular element potentially indicated that SMEs operated from a different knowledge base, such as when unpublished data were available to only a select few or alternatively indicated that few data were available to use in generating scores. In the second phase, SMEs were presented with a summary of the preliminary data for their element(s). In instances with a wide range of scores for a given element, the justification information was anonymized and redistributed to all the SMEs who scored that particular element, with the invitation to reconsider or confirm their initial score (Figure 1). Results for the IRAT average point scores were used to calculate an overall virus risk score for each of the 2 risk questions. Generally, virus scores of 1–3 were considered a low potential risk by the IRAT; scores of 4–7 were moderate; and scores of 8–10 were potential high risk. Scores at the boundaries of those ranges are described by a combination term such as moderate-high for a score of 7.5.

Figure 1.

Individual subject-matter expert point scores by element for the May 2017 scoring of influenza A(H7N9) virus, A/Hong Kong/125/2017, based on risk element definitions. Circles indicate individual point scores; circle sizes (examples…

Establishment of a Point of Reference

As a point of reference for low-risk viruses with respect to both IRAT questions, potential risk for emergence and risk for potential impact, SMEs evaluated the North America avian influenza A(H1N1) virus, A/duck/New York/1996. As expected, this virus received low risk scores from the SMEs; the summary average risk score was 2.3 (i.e., low risk) to achieve sustained human-to-human transmission. Similarly, the average risk score for the virus to substantially impact public health if it were to achieve sustained human-to-human transmission was 2.4 (low risk).