Published ahead of print on June 12, 2008, doi:10.1164/rccm.200802-282OC
© 2008 American Thoracic Society doi: 10.1164/rccm.200802-282OC
Mortality Reduction with Influenza Vaccine in Patients with Pneumonia Outside "Flu" SeasonPleiotropic Benefits or Residual Confounding?1 Department of Public Health Sciences, School of Public Health, and the 2 Department of Medicine, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada Correspondence and requests for reprints should be addressed to Dean T. Eurich, Ph.D., Assistant Professor, School of Public Health, University of Alberta, 13-103 Clinical Sciences Bldg., Edmonton, AB, T6G 2G3 Canada. E-mail: deurich{at}ualberta.ca
Rationale: Observational studies suggest a 50% mortality reduction for older patients receiving influenza vaccination; some deem this magnitude of benefit implausible and invoke confounding by the "healthy user effect" as an alternate explanation. Objectives: To evaluate unrecognized confounding by hypothesizing the presence of a 50% mortality reduction with vaccination for patients with pneumonia outside of influenza season. Methods: Clinical, laboratory, and functional data were prospectively collected on 1,813 adults with community-acquired pneumonia admitted to six hospitals outside of influenza season in the Capital Health region (AB, Canada). Vaccination status was ascertained by interview and chart review. Outcome was in-hospital mortality. Influenza-vaccinated patients were matched to a nonvaccinated control using propensity scores, and then multivariable regression was used to determine the independent association between vaccination and mortality. Measurements and Main Results: The cohort consisted of 352 vaccine recipients and 352 matched control subjects. Most (85%) patients were 65 years or older, 29% had severe pneumonia, and 12% died. Influenza vaccination was associated with a 51% mortality reduction (28 of 352 [8%] died vs. 53 of 352 [15%] control subjects; unadjusted odds ratio [OR], 0.49; 95% confidence interval [CI], 0.30–0.79; P = 0.004) outside influenza season. Adjustment for age, sex, and comorbidities did not alter these findings (adjusted OR, 0.45; 95% CI, 0.27–0.76). More complete adjustment for confounding (e.g., functional and socioeconomic status) markedly attenuated these benefits and their statistical significance (adjusted OR, 0.81; 95% CI, 0.35–1.85; P = 0.61). Conclusions: The 51% reduction in mortality with vaccination initially observed in patients with pneumonia who did not have influenza was most likely a result of confounding. Previous observational studies may have overestimated mortality benefits of influenza vaccination.
Key Words: influenza • bias • prospective study • pneumonia
Influenza-related complications, especially pneumonia requiring hospitalization and death in older adults, are common and costly (1, 2) and may be largely preventable with the wide-scale use of influenza vaccination. There remains a high degree of uncertainty, however, regarding the overall benefits of routine influenza vaccination for the elderly (3–5). Disparate opinions among patient advocacy groups, researchers, and policy makers have further fueled this controversy. In part, this debate has arisen because of the near absence of valid randomized trials with clinical endpoints whose results might be directly applicable to the elderly. Most of the evidence evaluating the clinical benefit of influenza vaccines has been derived from observational data (3, 6–8). Such observational studies conducted in older adults over multiple influenza seasons repeatedly suggest that inactivated trivalent influenza vaccine (hereafter, influenza vaccine) is able to reduce influenza-like illness, admissions to hospital for pneumonia, and all-cause mortality (3–8). Counterintuitively, the greatest benefit seen in observational studies is with reduction in this last endpoint—on the order of a 50% reduction in all-cause mortality (3, 6–8). A very large all-cause mortality benefit associated with influenza vaccination has important clinical, health/economic, and policy implications for recommendations in guidelines and resource allocation. For a number of reasons, however, experts have recently suggested that the magnitude of benefit seen with influenza vaccination in observational studies is implausible (4, 5, 9–11). First, although few randomized trials have been completed, no trial data support a mortality benefit with influenza vaccination (3–5). Second, over the last two decades in the United States, even while vaccination rates among the elderly have increased from 15 to 65%, there has been no commensurate decrease in hospital admissions or all-cause mortality (1–3). In fact, both admission rates and mortality in those 65 years and older have increased with increasing vaccine coverage over time (1–3). Of course, it is plausible that secular changes in multiple unmeasured patient factors during this time period of increased vaccination might have still led to increased rates of hospitalization and mortality. Third, some studies have observed mortality reduction with influenza vaccination in the "off-season" (i.e., the time of year when there is little if any circulating virus) (5, 9, 10). Last, studies that are able to better adjust for health and functional status or other measures of frailty demonstrate attenuation or abolition of mortality benefit, suggesting substantial residual confounding in most reported observational studies of vaccine effectiveness (4, 5, 11). This phenomenon has been variously referred to as frailty bias or the healthy-vaccinee effect or, more generally, as the "healthy-user" effect (4, 5, 11–13). The healthy-user is postulated to be a patient who, despite the presence of various coexisting conditions, is relatively healthier and has a predilection for better lifestyle behaviors (e.g., diet and exercise), more health seeking and preventative activities (e.g., cancer screening or treatment with medications like statins), better adherence to medical advice and therapies, and greater likelihood to be vaccinated against pneumococcus and influenza (11–13). Most observational studies of influenza vaccination have not been able to adequately control for this healthy-user effect. We hypothesized that if the healthy-user effect was responsible for the mortality benefit associated with influenza vaccination seen in observational studies, there should also be a significant mortality benefit present during the off-season. Because serious influenza episodes in the elderly usually lead to admission to hospital for lower respiratory tract infection, we took advantage of a previously assembled large and clinically detailed population-based cohort of consecutive adults admitted to hospital with pneumonia during the off-season to test our hypothesis. Our database was particularly well suited to examination of these questions because we prospectively collected vaccination status on all patients and we had a number of rarely collected potential indicators of the healthy-user effect (13).
The study and data collection methods have been previously described in detail (13–15). Briefly, between 2000 and 2002, all six hospitals within the Capital Health region implemented a previously validated and efficacious clinical pathway for the management of community-acquired pneumonia (CAP) (13–15). Capital Health is the largest integrated health system in Canada, serving over 1 million people within Edmonton, Alberta, Canada. During this period, data were prospectively collected on all 3,415 patients older than 17 years who were admitted to any hospital within the region for CAP (13–15). The only patients excluded from the population-based cohort were those who had tuberculosis or cystic fibrosis; those who were immunocompromised; or those who were pregnant. Then, for purposes of the present study, we excluded patients admitted to hospital during influenza season. National and regional laboratory surveillance data were used to define each influenza season by identifying the first and last occurrence of influenza isolated within the region. To evaluate the effect of influenza vaccination in the off-season, all 1,602 patients admitted to hospital for CAP during the so-defined 10 months of flu season were excluded from analyses of outcomes (i.e., between the months of November 2000 and March 2001 and between January 2002 and May 2002). Thus, all patients admitted to hospital between April to December 2001 and June to November 2002 were included in the present study. The study was approved by the Health Research Ethics Board of the University of Alberta.
Data Collection
Influenza Vaccination
Outcomes
Analysis Using this matched sample, we then completed a series of multivariable logistic regression analyses to evaluate the association between influenza vaccination and mortality. First, we calculated unadjusted estimates of mortality benefit. Second, we conducted simple adjustment using only age and sex. Third, we undertook what would be considered a "typical" administrative database analysis, adjusting for age, sex, admission from a nursing home, comorbidities, and number of medications. Last, we completed an analysis that incorporated clinical and laboratory data (e.g., PSI according to five risk classes) as well as data that are not routinely available in most administrative databases or retrospective chart reviews and might reflect healthy-user status (e.g., functional status, smoking status, pneumococcal immunizations). Some might question our a priori modeling strategy with respect to adjustment for PSI, a strategy that has been routinely advocated by us (14) as well as others (18, 19) when examining the potential benefits of vaccination. Given that the PSI is such an excellent risk-adjustment tool for pneumonia-related mortality, it seems to us inappropriate not to include it in models examining pneumonia-related mortality regardless of whether or not it is associated with (a potential confounder of) vaccine receipt (20). By analogy, male sex is well known to substantially increase the risk of pneumonia-related mortality (21), although it is not consistently associated with vaccine receipt—it would be difficult if not impossible to justify not including male sex in a multivariable model examining pneumonia mortality. Furthermore, a substantial proportion of the PSI score can be considered, in aggregate, a measure of chronic ill health (e.g., demographics, comorbidities) that was present "before" the episode of pneumonia occurred. Thus, it may also capture by proxy potential attributes related to qualities present in healthy-users, patients who are relatively healthier and more likely to seek out and undertake various screening activities as well as initiate preventive treatments such as use of statins or immunizations (12). We report unadjusted and adjusted odds ratios (ORs) from unconditional logistic regression models with their respective 95% confidence intervals (95% CIs) and associated P values. All analyses were conducted using SPSS version 14 (SPSS, Inc., Chicago IL) and Stata Intercool version 9 (StataCorp LP, College Station, TX). Similar analytic approaches were undertaken for the outcomes of ICU admission and the composite endpoint of death or ICU admission.
Sensitivity Analyses
Clinical Characteristics During 2000 to 2002, 1,813 patients 17 years or older were admitted to hospital for CAP during the off-season. Overall, admission to hospital occurred evenly throughout the off seasons: 353 (50%) of patients were admitted within 2 months of the start or end of influenza season and 351 (50%) were admitted during the remaining months; similarly, 303 (43%) patients were admitted after influenza season and 401 (57%) were admitted before the next flu season. Influenza vaccination status was documented for all patients: 370 (20%) had been vaccinated. As expected, influenza vaccination rates increased across quintiles of increasing propensity score, representing an increased predicted probability of receiving influenza vaccination: 4% versus 13% versus 21% versus 27% versus 35% of patients (P value for trend < 0.001). Using the propensity score, 352 of 370 influenza-vaccinated patients were matched to 352 unvaccinated patients, representing a 95% success rate (i.e., no match was available for 18 vaccinated subjects). The median age of the matched sample was 78 years, 54% were male, 28% were from nursing homes, 38% had chronic obstructive pulmonary disease, and 29% had severe (PSI class V) pneumonia. Patients were well matched and there were no statistically significant or clinically important differences in patient characteristics according to influenza vaccination status—with the exception of pneumococcal vaccination which was not part of the propensity score model and which was noted significantly more often among those vaccinated for influenza (Table 1).
Mortality Overall, 81 (12%) patients died in hospital. In unadjusted analyses, patients receiving influenza vaccination were significantly less likely to die compared with nonvaccinated patients (28 [8%] vs. 53 [15%]; OR, 0.49; 95% CI, 0.30–0.79; P = 0.004) (Figure 1). Using sequential multivariable models, the adjusted OR for the use of influenza vaccination and in-hospital mortality changed to 0.48 (95% CI, 0.30–0.79; P = 0.003) after adjustment for age and sex to 0.45 (95% CI, 0.27–0.76; P = 0.003) after the typical adjustments possible in large administrative databases (Figure 2). Subsequent models adjusting for clinical data not routinely available in administrative datasets yielded substantially different results. Inclusion of detailed clinical and laboratory data decreased the adjusted OR for death to 0.52 (95% CI, 0.30–0.90; P = 0.02) and, with full adjustment for all available data including functional status, need for an advance directive, pneumococcal immunizations, and socioeconomic status, yielded a much smaller and nonsignificant adjusted OR for death of 0.81 (95% CI, 0.35–1.85; P = 0.61; Figure 2 and Table 2). Similar results were observed for a more parsimonious model that included only age, sex, severity of pneumonia, functional status, smoking status, need for an advanced directive, and pneumococcal immunizations (OR, 0.79; 95% CI, 0.35–1.75; P = 0.56).
ICU Admission or Death Compared with those who were not vaccinated, patients who received influenza vaccine were less likely to be admitted to the ICU (Figure 1), regardless of model adjustment (unadjusted OR, 0.08; 95% CI, 0.02–0.26; P < 0.001; fully adjusted model OR, 0.17; 95% CI 0.04–0.71; P = 0.014). For the composite endpoint of ICU admission or death (Figure 1), patients receiving influenza vaccination were significantly less likely to be admitted to the ICU or die compared with nonvaccinated patients (unadjusted OR, 0.33; 95% CI, 0.21–0.52; P < 0.001). Fully adjusted models still suggest a trend toward a reduction in this composite endpoint for patients receiving influenza vaccination compared with those not vaccinated (OR, 0.50; 95% CI, 0.25–1.00; P = 0.05).
Sensitivity Analyses Our final sensitivity analyses related to an unmeasured and hypothetical single confounder in the unvaccinated control group. A confounding variable present in only 10% of control subjects that increased the relative risk of death by 3 or more would be entirely sufficient to explain the adjusted (albeit nonsignificant) 19% relative mortality reduction we observed. Similarly, an unmeasured confounder present in 20% of control subjects would only need to increase the risk of death by 2 to abolish the residual 19% mortality benefit of influenza vaccine seen in our analyses. In the overall parent dataset, differences in variables such as impaired functional status and need for an advance directive approach these hypothetical boundaries with respect to differential prevalence and risk of mortality.
We observed a large mortality benefit in patients who had received influenza vaccination before their hospitalization for pneumonia, even though it is extremely unlikely they had an influenza-related illness. Specifically, in unadjusted analyses, we found a statistically significant 51% relative reduction in all-cause mortality during hospitalization, from 15 to 8%, for those who were previously vaccinated. However, with progressively more careful adjustment for disease severity and measures of the healthy-user effect, the estimated benefit of influenza vaccination was markedly attenuated (19% reduction in mortality) and no longer statistically significant (P = 0.61). Our results are most consistent with residual and difficult-to-correct confounding. The initial (unadjusted) 51% reduction in all-cause mortality that we observed is consistent with what would be expected on the basis of previous observational studies that suggested mortality reduction on the order of 48 to 52% with influenza vaccination (3–8). To our knowledge, there have only been two studies that have previously specifically examined outcomes related to influenza vaccination in patients hospitalized with pneumonia (18, 19). In the first study, Herzog and colleagues studied 12,566 Medicare beneficiaries hospitalized with pneumonia during one influenza season; 61% of patients had unknown vaccination status, but most likely were not vaccinated (19). Thirty-day mortality was 24% among those who had been vaccinated before admission versus 32% among those not vaccinated (adjusted 35% reduction in mortality with vaccination, P < 0.001) (19). In the second study, Spaude and coworkers examined 17,393 adults hospitalized during four consecutive influenza seasons; 47% of patients had unknown vaccination status (18). In-hospital mortality was 7%, and influenza vaccination was associated with an adjusted 43% mortality reduction (P < 0.001) (18). Although both of these studies were rigorously conducted, they share three common limitations (18, 19): (1) ascertainment of vaccination status was incomplete, (2) data from the influenza off season were excluded from all analyses, and (3) neither study had information available with respect to functional status or other indicators of the healthy user. These last two limitations are common to almost all published observational studies of the effectiveness of influenza vaccination (5–10). Given that we were able, to some degree, to overcome a number of limitations endemic to previous studies, we believe that our results empirically demonstrate that the mortality benefits of influenza vaccination may have been largely overestimated. That is, we observed that a striking and highly significant association became attenuated and nonsignificant after accounting for disease severity and measures of functional status. The same confounding by the healthy-user effect has been implicated in observational studies that found reductions in the risk of myocardial infarction or stroke with hormone therapy (12), as well as the noncardiovascular "pleiotropic effects" of preventive medications such as statin therapy (13). Moreover, even previous studies of the mortality benefits of influenza vaccination in elderly patients have suggested potential confounding by the healthy-user effect that cannot be adjusted for using routine administrative data, possibility due to the fact that various groupings of administrative data are relatively poor markers for disease severity or overall frailty (9, 11). Alternately, because our estimate of mortality reduction with influenza vaccination is still consistent with a benefit (adjusted OR, 0.81; 95% CI, 0.35–1.85; P = 0.61), our data could also be interpreted as demonstrating an unanticipated pleiotropic benefit of influenza vaccination. It has been posited that even in patients with bacterial pneumonia, influenza vaccination might reduce the severity of illness by one of two mechanisms. Vaccination could improve outcomes either through augmentation of innate immunity (24) or by virtue of the fact that vaccination prevents influenza infection (which can itself predispose individuals to serious bacterial infections with Streptococcus pneumoniae or Staphylococcus aureus) and leads to a shift in the microbial spectrum toward far less virulent pathogens (18). Despite such speculations, we believe that our results are most consistent with difficult-to-control confounding. This observational study has at least four major limitations. First, because we studied only patients hospitalized with pneumonia, our findings may not necessarily be generalized to broader population-based studies, which tend to include much healthier and lower risk patients who never become ill enough to require admission. Although this might be expected to lead to differences in baseline risk and absolute mortality rates, there is no reason to believe that the relative benefits of vaccination should be different in our study population (25, 26). Second, we did not have any measures of immunity and very limited data regarding etiology, and so it is possible that some patients we studied did actually have influenza infection. However, we restricted analyses to time periods when influenza was not circulating in the community and we conducted a sensitivity analysis that extended the beginning and end of influenza season by 1 month in each direction and this did not materially affect our results. Third, we only have information related to outcomes during hospitalization, although most of the morbidity and eventual mortality associated with influenza infection in the elderly occurs during hospital admission (5, 7, 8, 18, 19). Last, although we had more clinical data than most studies, our measures of the healthy-user effect were fairly rudimentary and limited. Ideally, we would also have had more detailed information about socioeconomic status; health care visits before illness; habits related to diet, alcohol intake, regular exercise, and health screening and promotion activities; and, especially, adherence to prescribed medications (12, 13). In conclusion, the fact that we were able to find such a large and fairly robust mortality benefit with influenza vaccination in patients hospitalized with pneumonia during the off-season implies that studies that have restricted their analyses to the influenza season have overestimated the potential mortality benefit of vaccination. If this is the case, it has important implications for researchers (e.g., need for better vaccines and need for better observational study designs) and for policy makers (e.g., different guideline recommendations and more rational resource allocation strategies). Many initiatives are underway to increase rates of annual influenza vaccination in the elderly because of the putative mortality benefits, despite increasing concerns about the evidence underpinning these well-intended recommendations (27). It would seem prudent to us that before the implementation of such overarching recommendations, that higher quality evidence of benefit be generated first. Furthermore, because we were also unable to control completely for the healthy-user effect, we hope our findings might help tilt the balance toward clinical equipoise and permit much needed and adequately powered randomized trials of influenza vaccine in the elderly to take place.
For their dedication and diligence, the authors thank the Community-Acquired Pneumonia Pathway research nurses and the staff of EPICORE (University of Alberta) for data management services.
Supported by establishment grants from Alberta Heritage Foundation for Medical Research (AHFMR); grants-in-aid from Capital Health; and unrestricted grants from Abbott Canada; Pfizer Canada, and Jannsen-Ortho Canada (all to T.J.M.). D.T.E. receives a salary support award from AHFMR (Population Health Investigator). S.R.M. receives salary support awards from AHFMR (Health Scholar) and the Canadian Institutes of Health Research (CIHR-New Investigator). The funding sources had no role in study design, data collection, data analysis or interpretation, or writing of the report. Originally Published in Press as DOI: 10.1164/rccm.200802-282OC on June 12, 2008 Conflict of Interest Statement: None of the authors has a financial relationship with a commercial entity that has an interest in the subject of this manuscript. Received in original form February 17, 2008; accepted in final form June 6, 2008
Related articles in AJRCCM:
This article has been cited by other articles:
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
TOP
ABSTRACT
AT A GLANCE COMMENTARY
