Published ahead of print on May 29, 2008, doi:10.1164/rccm.200801-131OC
© 2008 American Thoracic Society doi: 10.1164/rccm.200801-131OC
Interleukin-8 as a Stratification Tool for Interventional Trials Involving Pediatric Septic Shock1 Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center and Cincinnati Children's Research Foundation, and Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, Ohio; 2 Division of Critical Care Medicine, Children's Hospital and Research Center Oakland, Oakland, California; and 3 Eli Lilly Research Laboratories, Indianapolis, Indiana Correspondence and requests for reprints should be addressed to Hector R. Wong, M.D., Division of Critical Care Medicine, Cincinnati Children's Hospital Medical Center, 3333 Burnet Avenue, Cincinnati, OH 45229-3039. E-mail: hector.wong{at}cchmc.org
Rationale: Interventional clinical trials involving children with septic shock would benefit from an efficient preenrollment stratification strategy. Objectives: To test the predictive value of interleukin (IL)-8 for 28-day mortality in pediatric septic shock. Methods: A training data set (n = 40) identified a serum IL-8 of greater than 220 pg/ml as having a 75% sensitivity and specificity for predicting 28-day mortality. This cutoff was then subjected to a series of validation steps. Measurements and Main Results: Subjects were drawn from two large, independent pediatric septic shock databases. Prospective application of the IL-8 cutoff to validation data set 1 (n = 139) demonstrated 78% sensitivity and 64% specificity for 28-day mortality. A serum IL-8 level of 220 pg/ml or less, however, had a negative predictive value for 28-day mortality of 95% in validation data set 1, which was subsequently applied to an independently generated data set of children with septic shock (validation set 2, n = 193). A serum IL-8 level of 220 pg/ml or less had a negative predictive value for 28-day mortality of 94% when applied to validation set 2. Conclusions: A serum IL-8 level of 220 pg/ml or less, obtained within 24 hours of admission, predicts a high likelihood of survival in children with septic shock. We propose that IL-8 can be used to exclude such patients from interventional clinical trials and ultimately derive a study population with a more favorable risk to benefit ratio when subjected to a study agent.
Key Words: pediatrics • septic shock • biomarkers • interleukin-8 • stratification
Septic shock continues to be a major child health issue, leading to over 4,000 childhood deaths per year in the United States alone (1, 2). Apart from antibiotics, pediatric intensive care unit (PICU)–based supportive care, and vaccination strategies, there are no specific approved therapies for pediatric septic shock. Until the recent RESOLVE (Researching severe Sepsis and Organ dysfunction in children: a gLobal perspectiVE) trial (3), progress in the field has been hampered by a lack of large randomized trials specifically dedicated to children with septic shock. Conducting randomized trials involving children with septic shock is a difficult endeavor for two broad, interrelated reasons. First, septic shock is a syndrome rather than a distinct disease entity, and is thus characterized by broad heterogeneity with regard to etiology, physiologic consequences, morbidity, and mortality (2, 4). Consequently, defining an optimal population for interventional clinical trials has been challenging, almost to the point of being not feasible. Second, compared with the adult septic shock population, the pediatric septic shock population is relatively smaller, as is the associated mortality rate (1, 5). Consequently, enrolling a sufficient number of patients, based on current stratification and eligibility criteria, can be prohibitive in terms of achieving sufficient power to demonstrate efficacy. One potential solution for achieving sufficient statistical power is to involve a large number of pediatric centers; the recent RESOLVE trial involved 104 pediatric centers from 18 countries (3). Although this strategy holds potential for a valid outcome having broad relevance, it also has the potential to introduce additional patient heterogeneity and high variability of "standard" therapy, both of which can be profound confounders for bringing successful trials to fruition (4). Given these challenges, interventional clinical trials involving children with septic shock would significantly benefit from an effective, clinically feasible stratification strategy. The goal of this stratification strategy would be to identify patient populations having knowable risks for either poor or good outcome as a means for improving the risk to benefit ratio of a given intervention (4). If this goal is attainable, then it may become more feasible to conduct interventional clinical trials in children with septic shock using mortality as the primary outcome. In an ongoing translational research program involving microarray-based, genomewide expression profiling in children with septic shock, we recently reported that interleukin (IL)-8 mRNA levels, measured within 24 hours of presentation to the PICU, were increased in nonsurvivors of pediatric septic shock (6). These microarray data were corroborated by increased IL-8 protein in the parallel serum samples of nonsurvivors, thus raising the possibility that serum IL-8 can be used as a biomarker for stratifying mortality risk in children with septic shock. The primary aim of the current study is to formally and prospectively test the hypothesis that a serum IL-8 level, measured within 24 hours of admission to the PICU, is a robust biomarker for predicting the outcome of children with septic shock. The strategy for testing this hypothesis involves the prospective use of independent training and validation data sets.
Genomics of Pediatric Septic Shock Database The Genomics of Pediatric Septic Shock (GPSS) database was used to generate the training data set and validation data set 1. The GPSS database has been previously described in detail (6–8). Briefly, this noninterventional database supports a translational research program focused on genome-level expression profiles (microarray-based) of children with septic shock. Eighteen PICUs in the United States have contributed samples to the GPSS (see ACKNOWLEDGMENT). The database contains extensive clinical data, RNA samples, and concomitant serum samples. RNA and serum samples are collected within 24 hours of admission to the PICU and 48 hours after the initial sample collection. In the current analyses, we used only the serum samples drawn within 24 hours of admission to the PICU. The subjects enrolled in the GPSS database include children (<10 yr of age) admitted to the PICUs of multiple institutions and having a diagnosis of septic shock. Pediatric-specific criteria are used to classify patients as having septic shock (9), and the only exclusion criteria are the inability to obtain informed consent or a white blood cell count of less than 1,000 cells/mm3. Standard clinical care is not under protocol for the enrolled subjects, and none of the subjects in the GPSS database are duplicated in the RESOLVE database (see below). Patients are monitored for 28 days to determine survival. Accordingly, all mortality-related data in this report refer to this 28-day period.
RESOLVE Database The study protocols supporting the GPSS and RESOLVE databases were approved by the individual institutional review boards of each participating institution. The patients in both databases are heterogeneous with regard to infecting organisms and comorbidities.
IL-8 Measurements
Analytical Approach
Specificities, sensitivities, positive predictive values, and negative predictive values were calculated by standard methods based on contingency tables. Confidence intervals for contingency table-derived data and likelihood ratios were calculated using the VassarStats website (http://faculty.vassar.edu/lowry/clin1.html). Continuous variables were compared by Student's t test. Categorical variables were compared by
Training Data Set The first step in testing the ability of IL-8 to serve as biomarker for outcome in pediatric septic shock was generation of an ROC. The previously reported cohort of children with septic shock (32 survivors and 8 nonsurvivors) served as the training data set for ROC derivation (6). Serum IL-8 levels for this analysis, and all subsequent analyses, were obtained within 24 hours of admission to the PICU. Figure 1 demonstrates the ROC generated from this 40-patient cohort. From this ROC, we extrapolated that a serum IL-8 concentration greater than 220 pg/ml had 75% sensitivity (95% confidence interval [CI], 36–96%) and 75% specificity (95% CI, 56–88%) for predicting 28-day mortality in this training cohort. The positive and negative predictive values for mortality were of 43% (95% CI, 19–70%) and 92% (95% CI, 73–99%), respectively. This IL-8 concentration cutoff of 220 pg/ml was consistently used in all subsequent analyses, without post hoc calibrations.
Validation 1 We next sought to test the ability of a serum IL-8 concentration greater than 220 pg/ml to predict mortality for all patients in the GPSS database (validation data set 1). Validation data set 1 consisted of the 40 patients described above in the training data set, plus an additional 99 consecutive patients with septic shock (10 nonsurvivors) enrolled in the GPSS database, but not previously reported (139 total patients). Table 1 demonstrates that the patients in the training data set and validation data set 1 were similar with respect to mortality rates, age, severity of illness (PRISM [Pediatric Risk of Mortality] score), sex distribution, and median IL-8 levels measured within 24 hours of admission to the PICU. Figure 2 represents the contingency table for patients in validation data set 1 having survivorship and serum IL-8 concentration as the contingency variables. A serum IL-8 concentration greater than 220 pg/ml had 78% sensitivity, 64% specificity, and 25% positive predictive value for mortality. In contrast, a serum IL-8 of 220 pg/ml or less had a negative predictive value for mortality of 95%. In other words, a serum IL-8 concentration of 220 pg/ml or less predicted survival with 95% accuracy in validation data set 1.
This analysis demonstrates that a serum IL-8 level greater than 220 pg/ml, measured within 24 hours of admission to the PICU, does not serve as a robust biomarker for predicting mortality in children with septic shock, at least in this cohort of patients. Importantly, however, when this same measurement generates a serum IL-8 concentration of 220 pg/ml or less, it may have the potential to predict survival with 95% certainty.
Validation 2 Table 2 compares demographic data between patients in validation data set 2, the patients in the RESOLVE database 10 years or older with available IL-8 data, and the entire RESOLVE database. As would be predicted, patients in the RESOLVE database 10 years or older with available IL-8 data were older than the two other comparison groups. Otherwise, the three groups were similar with respect to mortality, illness severity, and sex distribution, thus indicating that the subgroup of patients drawn for validation data set 2 are relatively representative of the entire RESOLVE database.
Figure 3 represents the contingency table for validation data set 2 having survivorship and IL-8 concentration as the contingency variables. A serum IL-8 concentration of 220 pg/ml or less had a negative predictive value of 94% for mortality when prospectively applied to validation data set 2. This analysis demonstrates that a serum IL-8 concentration of 220 pg/ml or less, obtained within 24 hours of admission to the PICU, predicts survival with 94% certainty when prospectively applied to an independently generated cohort of children with septic shock.
Performance across All Patients In this analysis, we combined all patients in validation data set 1 and validation data set 2 (332 total patients, 44 nonsurvivors) and generated a contingency table using the same conventions described above (Figure 4). A serum IL-8 concentration greater than 220 pg/ml had 82% sensitivity, 51% specificity, and 20% positive predictive value for mortality. In contrast, a serum IL-8 concentration of 220 pg/ml or less had a negative predictive value for mortality of 95%. Thus, when all patients in validation data sets 1 and 2 are examined as a single cohort, a serum IL-8 concentration of 220 pg/ml or less predicts survival with 95% accuracy and a narrow CI (90–98%).
To further illustrate the robustness of IL-8 for patient stratification, we again combined all of the patients in validation data sets 1 and 2 and placed them into one of two groups post hoc: those with a serum IL-8 concentration of 220 pg/ml or less (n = 154) and those with a serum IL-8 concentration greater than 220 pg/ml (n = 178). As shown in Figure 5, the group having a serum IL-8 concentration of 220 pg/ml or less had a significantly lower 28-day mortality rate and a significantly lower odds ratio for mortality than the group having a serum IL-8 concentration greater than 220 pg/ml. This post hoc analysis further demonstrates the ability of serum IL-8 to stratify patients into two patient populations having significantly different risks for 28-day mortality.
A final post hoc analysis examined adverse events in the RESOLVE trial based on IL-8 stratification. As shown in Table 3, the number of adverse events, among patients younger than 10 years and with available IL-8 data, were not significantly different between patients with an IL-8 serum level of 220 pg/ml or less and patients with an IL-8 serum level greater than 220 pg/ml.
IL-8 belongs to the CXC chemokine family, is the major neutrophil chemoattractant and activator in humans, and is well known to play a role in the host response to infection-related conditions such as septic shock and pneumonia (16). The ability to readily measure IL-8 in human samples, combined with its well-defined biological role, has provided the foundation for investigations centered on IL-8 as a biomarker in sepsis and septic shock. Multiple studies have demonstrated the potential utility of IL-8 in the early diagnosis of systemic bacterial infections, particularly in neonates (17–21). Of more direct relevance to the current study, previous studies have correlated increased serum IL-8 levels with increased illness severity and mortality in adults and children with septic shock (22–34). To our knowledge, however, the negative predictive value of IL-8 in pediatric septic shock has never been tested in the manner and with the magnitude of the current study. On the basis of our previous report demonstrating increased IL-8 mRNA and protein in nonsurvivors of pediatric septic shock (6), we set out to test a singular hypothesis focused on the ability of serum IL-8 to serve as an outcome-specific biomarker. We used the initially reported cohort of patients as the training data set from which we derived a serum IL-8 cutoff value of 220 pg/ml. Importantly, we persisted with this particular value in all subsequent analyses, rather than deriving alternative post hoc values. When this serum IL-8 concentration was prospectively applied to a validation data set, generated by the same group of investigators (i.e., the GPSS database), the resulting sensitivities, specificities, and positive predictive values for mortality were not sufficiently robust for clinical utility. In contrast, the negative predictive value of this serum concentration was potentially very robust in that it could predict 28-day survival with 95% certainty. To test the veracity of this observation (i.e., high negative predictive value), we prospectively applied the serum IL-8 cutoff to an independently generated cohort of children with septic shock (the RESOLVE database). The effectiveness and significance of this approach deserves emphasis. The RESOLVE database was generated by a completely different group of investigators (3). Therefore, the RESOLVE database likely has characteristics that are unique relative to those of the GPSS database, and vice versa. In addition, IL-8 measurements for the RESOLVE database were conducted using a different technique than that of the GPSS database. The use of two different assay techniques reflects the fact that the two study protocols were independently conducted by two distinct research groups. As discussed above, although the two techniques generally have good correlation, the multiplex technique appears to be more reliable at lower IL-8 concentrations (11–15). Despite these potentially profound confounders, we demonstrated that a serum IL-8 concentration of 220 pg/ml or less predicted survival with 94% accuracy in the RESOLVE database patients, thus considerably increasing the reliability and generalizability of our observations. Because overall mortality for pediatric septic shock has been estimated to be approximately 10% (1, 5), and the overall patient cohort in this study had a mortality rate of 13%, one could argue that predicting survivorship with 95% certainty has limited utility. What then is the clinical utility of a serum IL-8 concentration of 220 pg/ml or less in children with septic shock? For the care of an individual child with septic shock, we do not envision any particular utility. Rather, we envision that this cutoff could be used as a stratification strategy for future interventional clinical trials involving children with septic shock. Although previous biomarker-based stratification strategies have focused on a priori selection, for inclusion of patients with the highest mortality risk (4, 35–37), the strategy that we are envisioning has a subtle, but important difference. The strategy we envision would exclude patients who would otherwise be eligible for enrollment, but who have a low mortality risk based on a study entry serum IL-8 concentration of 220 pg/ml or less. This type of strategy would serve to exclude patients who have a high probability (up to 95%) of surviving septic shock with standard care, thus ultimately generating a study population with a higher mortality rate (20% in the current cohort of patients). We are not aware of any other septic shock biomarker that can achieve a 95% negative predictive value for mortality. Other biomarkers (i.e., IL-6), or a combination of biomarkers, may have a similar (or better) negative predictive value, but this hypothesis has not been directly tested in the manner described in the current study. By excluding patients with a low mortality risk, who would otherwise meet entry criteria, the risk to benefit ratio could be improved such that experimental interventions may have a higher likelihood of demonstrating efficacy. This assertion is predicated on an experimental intervention having a relatively constant risk profile across the range of illness severity. Conversely, the stratification strategy may not be as robust if the potential for harm of the experimental strategy significantly increases in proportion to illness severity. For example, of the 193 patients in the RESOLVE trial used in the current analysis, 72 (37%) would have been excluded under the current proposal based on study entry IL-8 levels. These 72 patients otherwise had a sufficient level of illness severity to warrant enrollment. These same 72 patients, however, had a mortality rate of 6% (compared with an overall mortality rate in the RESOLVE trial of 17%), whereas 5% of the overall study population in the RESOLVE trial had a study drug–related serious adverse event (3). In addition, our post hoc analysis of adverse events in the RESOLVE trial demonstrated similar adverse event rates in patients with serum IL-8 levels of 220 pg/ml or less and patients with serum IL-8 levels greater than 220 pg/ml. Thus, including these 72 patients could potentially expose them to a risk of a study drug–related serious adverse event that is virtually identical to that of the mortality risk. Several potential limitations of the current study deserve discussion. First, the IL-8 measurement techniques varied between the two databases, as discussed above. Second, the GPSS database was developed with the goal of generating genome-level expression data in children (6–8). The decision to limit the GPSS database to children younger than 10 years was a programmatic decision seeking to distinguish our efforts from that of other investigative groups studying adult septic shock. At the initiation of the GPSS database, we decided to only include patients younger than 10 years because this population would be exclusively prepubertal and would undeniably qualify as being representative of "children." All children younger than 10 years and meeting criteria for septic shock are eligible for enrollment in the GPSS, with the only exclusion criteria being the inability to obtain informed consent or a white blood cell count less than 1,000 cells/mm3, which would preclude isolation of sufficient RNA for microarray analysis. Patients receiving chemotherapy, undergoing bone marrow transplantation, and/or receiving immunosuppressive medications, regardless of their expected outcomes, are included in the GPSS database. In contrast, the RESOLVE database was developed with the goal of testing a novel therapeutic strategy in patients between 38 weeks of corrected gestational age and 17 years of age (3). As a test agent efficacy trial, the patient population was more highly selected compared with that of the GPSS database. For example, patients with high risk of intracranial bleeding, expected to die before the 28-day study period from preexisting conditions, with end-stage renal or liver disease, with a platelet count less than 30,000/mm3, and/or having undergone bone marrow transplantation were excluded from the RESOLVE trial, whereas these types of patients were not excluded from the GPSS database. Patients undergoing chemotherapy and patients receiving immunosuppressive medications were included in the RESOLVE trial as long as they did not meet the exclusion criteria listed above. As stated above, despite these potentially profound confounders, we were able to prospectively validate the negative predictive value of serum IL-8 using the RESOLVE database. The ability to overcome these potential limitations, while testing a single hypothesis, without post hoc modifications, strengthens the validity and generalizability of our data. A third potential limitation is that all available patients in the GPSS database were used, whereas only a selected group of patients from the RESOLVE database were used for the current study (i.e., patients < 10 yr of age and having available IL-8 data). The rationale for selecting patients from the RESOLVE database was to derive a validation data set with a comparable age range as that of the training data set, but this approach excluded 60% of the patients in the RESOLVE database. Thus, we recognize that our data may be generalizable only to children with septic shock who are younger than 10 years. A post hoc analysis, however, involving all 278 patients in the RESOLVE database having available IL-8 data (58% of the total RESOLVE database and including children up to 17 yr of age) demonstrated that a serum IL-8 level of 220 pg/ml or less had a negative predictive value for 28-day mortality of 92% (95% CI, 85–96%). In summary, we have demonstrated that a serum IL-8 concentration of 220 pg/ml or less, obtained within 24 hours of admission to the PICU, serves as a robust negative predictor for 28-day mortality in children younger than 10 years with septic shock. The assertion of robustness is supported by prospective validation using a patient cohort completely independent of the training cohort. Furthermore, the ability to validate this cutoff using an independent cohort implies that the cutoff is valid across a broad spectrum of children with septic shock (generalizability). Generalizability is also supported by the fact that the data are derived from children admitted to multiple pediatric centers and from children with significant comorbidities. We propose that a serum IL-8 level of 220 pg/ml or less, obtained within 24 hours of admission, can be used as an effective strategy to exclude patients from interventional clinical trials involving children with septic shock. Such a strategy would serve to exclude patients with a high likelihood of surviving with standard care and ultimately derive a study population with a more favorable risk to benefit ratio when subjected to a study agent. Because reagents to measure IL-8 are widely available and serum is a readily obtainable biological sample, it is highly likely that a rapid, point-of-care test can be developed for measuring IL-8 at the bedside. Thus, the proposed approach is highly feasible.
Supported by a grant from the National Institute of General Medical Sciences (RO1 GM064619). H.R.W. and the GPSS database are supported by the National Institutes of Health and the Cincinnati Children's Research Foundation. The RESOLVE database is funded by Eli Lilly. Originally Published in Press as DOI: 10.1164/rccm.200801-131OC on May 29, 2008 Conflict of Interest Statement: H.R.W. submitted a provisional patent application in August 2007 for the use of IL-8 as a stratification tool in children with septic shock. N.C. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. D.S.W. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. M.T.B. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. M.M. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. K.O. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. W.L.M. is an employee and shareholder of Eli Lilly and Company. M.D.W. is an employee and shareholder of Eli Lilly and Company. Contributing investigators and centers for the GPSS database are as follows: Geoffrey L. Allen (Children's Mercy Hospital, Kansas City, MO); Nick Anas (Children's Hospital of Orange County, Orange, CA); Paul Checchia (St. Louis Children's Hospital, St. Louis, MO); Allan Doctor (St. Louis Children's Hospital, St. Louis, MO); Robert Fitzgerald (Devos Children's Hospital, Grand Rapids, MI); Robert J. Freishtat (Children's National Medical Center, Washington, DC); Jose Gutierrez (Pediatric Critical Care of Arizona, Phoenix, AZ); Meena Kalyanaraman (Newark Beth Israel Medical Center, Newark, NJ); Gary Kohn (Morristown Memorial Hospital, Morristown, NJ); Cheri Landers (Kentucky Children's Hospital, Lexington, KY); Richard Lin (The Children's Hospital of Philadelphia, Philadelphia, PA); Gwenn McLaughlin (Jackson Memorial Hospital, Miami, FL); Keith Meyer (Miami Children's Hospital, Miami, FL); Scott Penfil (DuPont Hospital for Children, Wilmington, DE); Steve Shane (Washoe Medical Center, Reno, NV); Thomas P. Shanley (C.S. Mott Children's Hospital at the University of Michigan, Ann Arbor, MI); Neal J. Thomas (Penn State Children's Hospital, Hershey, PA); Nancy M. Tofil (The University of Alabama at Birmingham, Birmingham, AL); Margaret Winkler (The University of Alabama at Birmingham, Birmingham, AL); and Douglas Willson (University of Virginia, Charlottesville, VA). Received in original form January 21, 2008; accepted in final form May 19, 2008
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
TOP
ABSTRACT
AT A GLANCE COMMENTARY
2 or Fisher exact test when appropriate. When indicated, significance was set at 0.05. 
220 pg/ml), respectively. Note that an IL-8 serum level 
10 YEARS OF AGE WITH AVAILABLE IL-8 LEVELS, AND ALL PATIENTS IN THE RESOLVE DATABASE
