Published ahead of print on July 3, 2003, doi:10.1164/rccm.200302-199OC
© 2003 American Thoracic Society Decision Analysis of Antibiotic and Diagnostic Strategies in Ventilator-associated PneumoniaCenter for Pulmonary and Critical Care Medicine, Department of Laboratory Medicine, Pharmacy, and Finance, North Shore-Long Island Jewish Health System, Manhasset; New York University School of Medicine, New York; and SUNY at Stony Brook, Stony Brook, New York Correspondence and requests for reprints should be addressed to David E. Ost, M.D., Center for Pulmonary and Critical Care Medicine, North Shore University Hospital, 300 Community Drive, Manhasset, NY 11030. E-mail: dost{at}nshs.edu
The optimal strategy for ventilator-associated pneumonia remains controversial. To clarify the tradeoffs involved, we performed a decision analysis. Strategies evaluated included antibiotic therapy with and without diagnostic testing. Tests that were explored included endotracheal aspirates, bronchoscopy with protected brush or bronchoalveolar lavage, and nonbronchoscopic mini-bronchoalveolar lavage (mini-BAL). Outcomes included dollar cost, antibiotic use, survival, cost-effectiveness, antibiotic use per survivor, and the outcome perspective of financial cost–antibiotic use per survivor. Initial coverage with three antibiotics was better than expectant management or one or two antibiotic approaches, leading to both improved survival (54% vs. 66%) and decreased cost ($55,447 vs. $41,483 per survivor). Testing with mini-BAL did not improve survival but did decrease costs ($41,483 vs. $39,967) and antibiotic use (63 vs. 39 antibiotic days per survivor). From the perspective of minimizing cost, minimizing antibiotic use, and maximizing survival, the best strategy was three antibiotics with mini-BAL.
Key Words: ventilator-associated pneumonia • decision analysis • evidence-based medicine • bronchoscopy Ventilator-associated pneumonia (VAP) complicates the course of patients on mechanical ventilation in approximately 8–28% (1–4) of cases and is associated with a mortality rate of 24–50% (2, 5–11). It has been shown that appropriate initial antibiotic therapy is among the most important predictors of successful clinical outcome (12–19). Diagnostic testing strategies, although frequently leading to changes in therapy, have not been definitively shown to directly impact survival (17, 18, 20–22). This is because the information gained from such tests often comes too late to alter outcome. Thus, initial broad-spectrum antibiotic therapy in high-risk populations has been advocated. However, increasing antibiotic resistance patterns among intensive care unit pathogens, fostered by empiric-broad spectrum antibiotic regimens, represents a significant competing concern. Indeed, antibiotic use before the development of VAP is associated with an increased risk for potentially resistant gram-negative infections and Methicillin-resistant Staphylococcus aureus (MRSA) (4, 23–26). In addition, infection with these resistant pathogens is associated with poorer outcome. Balancing these two competing goals, namely the need for rapid treatment with antibiotics that cover a broad range of potential pathogens versus limiting unnecessary antibiotic use, remains one of the clinical challenges in dealing with VAP. The objective of this study was to define more precisely the trade-offs involved, as well as the impact various diagnostic and antibiotic selection strategies might have on these two competing goals. To do this, we performed a decision analysis of late-onset VAP.
Overview and Definitions We constructed a hypothetical cohort of immunocompetent patients in the intensive care unit, intubated for 7 days, with evidence of late-onset VAP based on Centers of Disease Control criteria of fever, purulent secretions, leukocytosis, and radiographic infiltrates, with an estimated mortality rate of 20% for use in a decision analysis model. The initial decision was whether to do a diagnostic test immediately. The diagnostic strategies evaluated were as follows: none (empiric treatment only), quantitative nonprotected endotracheal cultures, bronchoscopy, and nonbronchoscopic mini-bronchoalveolar lavage (mini-BAL). If a diagnostic test was done, it was assumed to take place as soon as the VAP syndrome was recognized so that there was no significant delay in initial antibiotic therapy (27). The second decision was how many initial antibiotics to give. From zero to three antibiotics could be chosen. Based on a literature review and American Thoracic Society guidelines, two antibiotics were used for the baseline analysis (13, 14, 22, 23, 27–30). Zero initial antibiotics represented expectant management, with subsequent treatment based on culture results that would arrive in 72 hours. One premise was that the specific antibiotics chosen would be based on American Thoracic Society guidelines, as adapted to local antibiograms, formularies, and intensive care unit pathways (31). Antibiotics would be given immediately after the patient met Centers of Disease Control criteria for VAP and would be continued until diagnostic test results returned. If pathogens were identified, antibiotics would be adjusted to cover any identified pathogens, and unnecessary antibiotics would be discontinued. If all cultures were negative and the patient had ongoing severe sepsis or was unstable, the physician would continue antibiotics. If the patient was stable, antibiotics would be discontinued (22, 32, 33). The model was designed to assess the cost-effectiveness of different diagnostic and treatment strategies for VAP. Two separate aspects of cost were considered: financial cost and amount of antibiotics used. Effectiveness was measured in terms of hospital survival.
The Decision Analysis Model
Probability Variables: Methodology and Criteria Probability variables that were included in the decision model are shown in Tables 1 and 2 . A more detailed description of the probability variables and their definitions is available in the online supplement in the METHODS.
Cost and Charges Cost-effectiveness analysis was from the institutional perspective. This was chosen because many of the costs incurred by the different strategies would have to be paid for by the hospital without significant changes in Medicare reimbursement based on the current diagnostic-related groups system. To determine costs, data and input were used from administration, materials operation, finance, nursing, respiratory therapy, pharmacy, and microbiology. Costs were based on hospital charges taken from the hospital charge disk master for fiscal year 2002 and adjusted for departmental cost/charge ratios using a method similar to prior cost-effectiveness studies in the area of pneumonia (34–36). Antibiotic use was considered another form of "cost" and was used in a separate "cost-effectiveness" analysis. These variables are summarized in Table 2.
Sensitivity Analysis In all sensitivity analyses, a three-step process was used. First, one-way sensitivity analysis was done on all variables, and tornado diagrams were analyzed. Second, a two- and three-way sensitivity analysis was conducted, starting with the most sensitive variables. Combinations of less sensitive variables were also explored when those variables interacted, for example, efficacy of late adequate treatment and sensitivity of bronchoscopy. Finally, analysis of extremes was done using "best case" and "worst case" scenarios that favored each of the strategies.
The decision variables being studied were the number of antibiotics in the initial regimen and whether a diagnostic strategy should be used. Overall, to achieve the simultaneous goals of minimizing antibiotic use and financial cost while maximizing survival, a trade-off had to be made between antibiotic use on the one hand and cost minimization and survival on the other. A strategy of more initial antibiotics was better in terms of survival, financial cost, and financial cost per survivor. Antibiotic use, as measured by antibiotic days per survivor, increased with this strategy. Although diagnostic testing did not significantly impact survival, it helped by minimizing antibiotic days per survivor and financial costs. An analysis for each of the outcomes of (1) survival, (2) financial cost, (3) financial cost per survivor, (4) antibiotic use, (5) antibiotic use per survivor, and (6) the combined perspective of financial cost–antibiotic use cost per survivor is presented later in this article. Additional data and sensitivity analysis are available in the online supplement.
Survival Outcomes
Financial Cost Outcomes For the outcome of minimizing financial cost, the best strategy was mini-BAL, followed by ETT aspirate, bronchoscopy, and finally empiric therapy with no diagnostic testing. The optimal strategy was sensitive to the number of initial antibiotics given, the average cost per antibiotic day, the length of antibiotic treatment, the sensitivity of diagnostic testing, and the cost of mini-BAL (Table 5) . When no (expectant management) or one initial antibiotic was used, the best strategy in terms of financial cost minimization was empiric treatment without diagnostic testing. For two or more antibiotics, the best strategy was a diagnostic test with mini-BAL or ETT aspirate. The magnitude of the difference between mini-BAL and ETT aspirate was relatively small ($12–41), with a slight advantage going to either strategy depending on willingness to withhold antibiotics, the number of initial antibiotics, and the sensitivity of the tests (Figures 1A, 2, and 3) .
Financial Cost per Survivor For the outcome of financial cost-effectiveness, the best strategy in the baseline scenario was mini-BAL or ETT aspirate because effectiveness was essentially the same in all four strategies and these had the lowest cost. When the number of antibiotics in the initial regimen was considered as a factor, the most cost-effective strategy was a three-antibiotic strategy with mini-BAL (Table 3 and Figure 2). The absolute difference between diagnostic testing strategies (mini-BAL, ETT aspirate, or bronchoscopy) was relatively small, but the difference between these and the empiric strategy without testing was significant. Subsequent sensitivity analysis was done using three antibiotics combined with mini-BAL as the comparator. The optimum cost-effective strategy of three antibiotics with mini-BAL was sensitive to the variables of bronchoscopy sensitivity (threshold value for bronchoscopy sensitivity, 0.99), sensitivity of ETT aspirate (threshold value for ETT aspirate sensitivity, 0.77), and mini-BAL sensitivity (threshold value, 0.73).
Antibiotic Use Costs
Antibiotic Use per Survivor
Financial Cost–Antibiotic Use–Survival Benefit Using this combined perspective, the optimal strategy for VAP depends on the willingness to pay, expressed in willingness to use antibiotics as well as dollars. Although traditional financial cost-effectiveness suggests that a three-antibiotic strategy combined with mini-BAL is most cost-effective, this graphic clarifies the tradeoffs made to achieve this in that an average of 26 antibiotic days per patient will be required to do this, as compared with the strategy of doing nothing. The incremental cost-effectiveness with respect to antibiotic use for the three-antibiotic with mini-BAL strategy is 200.8 antibiotic days per additional survivor compared with the no-antibiotic strategy. From the perspective of minimizing antibiotic use, this is less than the incremental cost-effectiveness for all of the other strategies except for bronchoscopy with three antibiotics. Thus, the three-antibiotic mini-BAL strategy meets the definition for extended dominance and is superior to all one- and two-antibiotic strategies, irrespective of diagnostic technique, in terms of both minimizing antibiotic use and financial cost per additional survivor. This requires a willingness to pay of approximately 201 antibiotic days per additional survivor and $40,000 per survivor.
Our analysis indicates that the best strategy overall in late-onset VAP is broad-spectrum three-antibiotic initial coverage combined with diagnostic testing. As prior studies have suggested, diagnostic testing alone has relatively little impact on the individual patient's survival (17, 18, 20, 28) but is cost-effective in that it limits cost by decreasing unnecessary antibiotic use. Although some experts have advocated this previously, there is still considerable controversy in this area, and diagnostic testing for VAP is infrequently used. Second, testing alone is not enough. It must be combined with broad-spectrum coverage initially to optimize cost-effectiveness. The model structures the problem so that critical variables are more readily identified, allowing future research to target those areas that are most likely to have the greatest impact on patient outcomes. In this model, we can see that of the variables amenable to physician intervention, it is improving antibiotic selection that will have the greatest impact. This can be approached from a variety of perspectives. One perspective would be to enforce rigid guidelines that follow American Thoracic Society criteria. However, this might result in a wide spectrum of possible outcomes, given regional variation in incidence and resistance (37). Therefore, following published guidelines is a questionable outcome measure. Another approach would be to evaluate and develop more rapid and sensitive diagnostic tests. This decision model is useful in that it provides a framework for evaluating the potential utility of such diagnostic tests and can help predict whether a new diagnostic test would be cost-effective. The model also makes transparent the tradeoffs being made in terms of antibiotic use, cost, and survival so that decision makers can develop a rational policy. Although broad-spectrum initial coverage combined with mini-BAL is the most financially cost-effective, there is a trade-off in terms of antibiotic use. When viewed from the perspective of limiting the development of resistance, this financially cost-effective outcome "costs" over 200 antibiotic days per additional survivor. This emphasizes the issue of sustainable probability of coverage. If antibiotics are used very broadly, the short-term survival will be good, but over the long term, resistance may develop more rapidly. Excessive restriction of antibiotic use will not be cost-effective and will lead to excess mortality in the present rather than in the future. The model makes clear that the goal is the maximally efficient use of antibiotics over time. We can use the model as a framework to assess how additional antibiotics affect outcome, essentially asking what is the "return" on our investment of additional antibiotics. This is the incremental cost-effectiveness concept, but with antibiotic use rather than dollars as the unit of cost. Our analysis suggests that increasing the number of initial antibiotics from one to three is efficient. A similar theoretical approach could be applied to other clinical problems but would probably result in a much different assessment of incremental cost-effectiveness (e.g., community-acquired pneumonia). This study supplements the information and analysis of other investigators who have used decision theory methods to study VAP (38, 39). A prior decision analysis favored withholding antibiotics for patients with clinically diagnosed VAP (38). However, their approach did not take into account the probability of adequate antibiotic coverage and the evidence that suggests that inadequate initial antibiotic coverage increases mortality and cost. Thus, prior models could not adequately portray the strategy of broad initial coverage with rapid tapering based on diagnostic testing and could not provide insight into one of the most pivotal questions, namely how many antibiotics to use initially and how probability of adequate coverage affects outcome. Prior models did include consideration of adverse drug reactions and the probability of future episodes of drug resistant VAP. However, when we integrated data on adverse drug reactions, the impact was always negligible in terms of cost and survival for all scenarios. It was therefore eliminated from the tree at the outset of the study. This is consistent with the findings from prior investigators (38). Although future episodes of potentially drug-resistant VAP were integrated into prior models, our review of the literature did not reveal any strong evidence that would allow accurately portraying the impact of treatment for VAP today on future incidence and outcome of VAP, especially considering how often antibiotics are used in the intensive care unit. In comparison to the work of prior investigators, our model also allows measurement of other outcomes, such as antibiotic use and cost, to be considered along with survival. Ideally, a single multiattribute utility function integrating these outcomes could be developed. One limitation of our study was that we could not define this single multiattribute utility function. This was not feasible because we could not convert antibiotic use today to the future probability of antibiotic resistance globally or put a dollar cost on the impact of antibiotic use on future resistance patterns given the available data. Thus, although a multiattribute utility function combining the three outcome variables of antibiotic use, cost, and survival (based on the premise of preferential independence and utility independence) is in theory possible, the paucity of published data makes this impractical. Our study has several limitations. Some investigators have advocated that antibiotics be stopped when protected specimen culture results are negative and patients do not demonstrate evidence of severe sepsis, but physicians are often reluctant to do so (22, 28, 29, 32, 33, 40, 41). If antibiotics can be stopped earlier, when quantitative cultures are negative and the clinical context is favorable, then the benefit of diagnostic testing strategies will be greater. Sensitivity analysis of antibiotic stopping rules was therefore used to model the wide spectrum of clinical practice. Even if a full course of therapy was always given in the setting of negative culture results (conservative stopping rule), the benefits of diagnostic testing outweighed the costs, provided that two or more antibiotics were used initially (additional sensitivity analysis can be found in the online supplement). Our study also did not address the potential value of direct examination of bronchoalveolar lavage for neutrophils and intracellular organisms (22). We chose not to do this initially because it is not that widely available, and therefore, the point estimates of sensitivity as compared with other diagnostic tests are not well described. In particular, we did not know how often there was discordance between smear and culture (i.e., gram-positives grow on culture but not in smear or vice versa), how often there was partial concordance (i.e., gram-positive in both but only culture grows a gram-negative), what reference standard would take precedence (presumably culture), and how much this information would change the probability of adequate coverage. In addition, we could not estimate the actual cost of doing the procedure from our time–motion studies because of our lack of institutional expertise in this area. Certainly knowing that a gram-positive organism is present would be highly useful if vancomycin was not routinely given. The utility of knowing that a gram-negative organism was present might be less. The time delay between testing and subsequent antibiotic adjustments might be improved significantly, and there might be significant antibiotic savings. Also, if antibiotics could be withheld very quickly (after one dose) when results were negative, the impact would be even larger. Another limitation of this model is the uncertainty in the relationship between the number of antibiotics used and probability of effective coverage. Estimates of the efficacy of various antibiotic regimens (13, 14, 16, 23, 42) depend heavily on accurate knowledge of underlying incidence and resistance patterns as well as the formulary of available antibiotics. However, although this relationship has been described based on empiric data, there are no mathematical models in the literature relating number of antibiotics used to probability of adequate coverage. Given the variation in etiology across treatment sites, this could vary widely based on local patterns (37). Sensitivity analysis can give insight into how this information impacts on outcome. A sensitivity analysis on the probability of adequate coverage for one, two, and three antibiotic regimens demonstrates that if the third antibiotic provides an absolute incremental benefit of approximately 5% above the two-antibiotic strategy (e.g., 80% vs. 75% coverage for three versus two antibiotic regimens), then three antibiotics will remain the most cost-effective alternative provided that diagnostic testing is used (see online supplement for a more detailed sensitivity analysis). Despite these limitations, important conclusions can still be drawn regarding optimal strategies for late-onset VAP. The majority of alternative strategies can be eliminated based on a careful analysis of our willingness to use antibiotics and basic cost-effectiveness principles of extended dominance (Figure 2), as most of the inferior strategies cost more and are less effective while using more antibiotics per additional survivor compared with the mini-BAL with three-antibiotic strategy. This reduces the complex strategy question to the more manageable question of broad antibiotic coverage with either mini-BAL or bronchoscopy. Antibiotics before diagnostic testing, especially recent antibiotics, can significantly decrease sensitivity (43). If sensitivity falls below the 70% range, diagnostic testing becomes less cost-effective. Therefore, the diagnostic strategy chosen must be feasible for rapid execution around the clock because several studies have demonstrated that delays in adequate antibiotic therapy lead to adverse outcomes. Given that bronchoscopy is not readily available around the clock, mini-BAL is probably the most feasible strategy and will be cost-effective.
The authors thank Stephan Kamholz for his editorial review and Elizabeth Murray for her assistance with the preparation of this article.
This article has an online supplement, which is accessible from this issue's table of contents online at www.atsjournals.org Conflict of Interest Statement: D.E.O. has no declared conflict of interest; C.S.H. has no declared conflict of interest; G.J. has no declared conflict of interest; C.G. received a research grant from Merck for $12,000 in 2002–2003 and is on Merck's Speakers Bureau, receiving $4,000 in 2003, and received $10,000 from Roche Pharmaceuticals for serving as an advisor and speaker; S.C. has no declared conflict of interest; E.K. has no declared conflict of interest; M.L. has no declared conflict of interest; R.I. has no declared conflict of interest; A.M.F. has no declared conflict of interest. Received in original form February 11, 2003; accepted in final form June 26, 2003
This article has been cited by other articles:
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
TOP
ABSTRACT
METHODS
