ARTICLE

TOP ABSTRACT ARTICLE REFERENCES

THIS OFFICIAL WORKSHOP REPORT OF THE AMERICAN THORACIC SOCIETY WAS APPROVED BY THE ATS BOARD OF DIRECTORS JUNE 2001

Keywords: cost-effectiveness; critical care; intensive care unit; outcomes research

See APPENDIX 2 for glossary of terms. A pressing issue in medicine today is the cost of care. However, many clinicians remain skeptical of economic analyses and are confused over terminology. This skepticism and confusion are not entirely unfounded. There are many types of economic evaluations and they are inconsistently performed and reported. Heyland and coworkers demonstrated that, of more than 1,000 studies that addressed intensive care costs, only three met minimal criteria for scientific rigor (1).

In 1996, the U.S. Public Health Service Panel on Cost-effectiveness in Health and Medicine (PCEHM) published guidelines for the conduct and reporting of economic analyses (2). The guidelines represented useful general principles, although the extent to which they could be followed within a specific area of medicine, such as critical care, was not addressed. The American Thoracic Society convened a panel to address economic analyses in critical care. The goals were to interpret the PCEHM guidelines in the context of caring for critically ill patients, to recommend a standardized approach to the conduct and interpretation of cost-effectiveness analyses (CEAs) in critical care, and to highlight areas for future research. This document presents the summary and recommendations of that panel and is intended as a guide for critical care practitioners, researchers, and policy analysts.

Methods

We began with a 4-day workshop convened May 28-31, 1998. The workshop included experts in critical care, health economics, quality of life and utility assessment, decision analysis and econometric modeling, ethics, statistics, evidence-based medicine, and clinical trial design (see APPENDIX 1). The workshop also included representation from the PCEHM. Four break-out groups were established: determining costs, determining effects, modeling, and implications. The workshop was conducted as a series of alternating plenary and break-out sessions. Each break-out group was charged with a set of predefined tasks and questions. Plenary sessions were used for each break-out group to present iterative progress reports and receive feedback. Minutes were kept, summarized, and distributed throughout the workshop. After the workshop, representatives from each break-out group produced draft summaries. These were circulated and revised within groups and forwarded to the chairs, who produced one executive summary document and redistributed it to all panelists for feedback. This design was more flexible than some other consensus techniques or systematic evidence appraisals because our goals required the integration of evidence and expert opinion from a wide array of disciplines.

Principles of Economic Assessment in Health Care

Full discussion of the principles and approaches to health care economic analyses is beyond the scope of this paper and the interested reader is referred elsewhere (5). Briefly, there are four types of economic analyses used to compare alternative technologies: cost minimization, cost benefit, cost effectiveness, and cost utility (Table 1) (6). Each compares the costs and clinical outcomes associated with alternative interventions but uses different approaches to measure the effects. Cost minimization is often used as the basis for decisions in the intensive care unit (ICU). It assumes each technology is equally effective and identifies the option associated with least cost. Cost benefit measures costs and consequences in the same units (e.g., expresses a life saved as a monetary gain), is difficult to conduct, and is now rarely used.

Cost-effectiveness and cost-utility analyses are the preferred approaches to evaluate medical care technologies today (5). A cost-effectiveness analysis (CEA) produces a ratio, such as the cost per year of life gained, where the denominator reflects the gain in health from a specific intervention (e.g., life-years gained, number of additional survivors, or number of pneumonias averted) and the numerator reflects the cost in dollars of obtaining that gain (5). Cost utility is a type of cost-effectiveness where effects are expressed as utilities, such as quality-adjusted survival, facilitating comparisons across different diseases and interventions (e.g., quality-adjusted life- years [QALYs]). From here forward, we use the standard terminology in which CEA refers to both cost-effectiveness and cost-utility analyses.

The core purpose of a CEA is to determine the value or trade-off of a therapy or program. In other words, for a therapy known to be effective (e.g., the therapy produces additional survivors), a CEA asks "what is the cost to achieve that effect (gain in survival)?" This is expressed as the ratio of the incremental, or additional, costs divided by the incremental effects. In conducting a CEA, alternative approaches and methodologies can lead to differences in the estimates of costs and effects.

The PCEHM Reference Case

One key contribution of the PCEHM was to recommend that future CEAs at a minimum produce a Reference Case, where the cost-effectiveness (CE) ratio is generated by a standardized approach to important elements of the analysis, including the perspective chosen, the determination of costs and effects, the study time horizon, and the assessment of uncertainty and sensitivity analyses. This standardized approach facilitates comparability among CEAs. For example, by comparing the Reference Cases from different CEAs, one can make inferences about whether one therapy for one disease has a better or worse CE ratio than another therapy used in another area of medicine.

General Considerations Regarding CEA in the Critically Ill

We identified both general and specific issues about critical illness and ICU interventions that, although not necessarily unique to this field, occur commonly and can affect one or more methodological aspects of CEAs (see Table 2).

Evidence for the effectiveness of critical care interventions is often lacking. The impacts of many pharmacologic, technologic, and programmatic interventions in the ICU are not established. Thus, their effects are often estimated or presumed, rather than known, and some CEAs simply assume that a patient not receiving intensive care would have died (10). Because the purpose of a CEA is to provide insight into the cost incurred per effect gained, a lack of evidence regarding effect diminishes the value of a CEA (11).

ICU care is often supportive, rather than curative. The goals of many ICU interventions are to stabilize and support patients (e.g., mechanical ventilation), rather than to cure or improve an underlying condition. In such instances, isolating the clinical and economic consequences of individual interventions can be difficult.

Critical illness is a complex process that can occur in varied, heterogeneous populations. ICU interventions are often applied to heterogeneous patient populations with different underlying comorbidities and probability of survival. ICU patients can also develop complications, which themselves have multisystem manifestations. Determining the effect of a particular therapy in such situations is difficult, complicating both clinical trials and CEAs.

ICU outcome measures are not well suited for CEA. ICU outcome measurements are often physiological parameters (e.g., arterial oxygenation or cardiac ejection fraction) or intermediate outcomes (e.g., ventilator-free days). The relationships of these endpoints to patient-centered outcomes, such as survival or quality of life, are not clear. Although survival is often used in ICU studies, the time horizon is usually short (e.g., 28 days). Such short follow-up complicates estimation of the long-term benefits (e.g., years of life gained) of many ICU therapies.

Recommended outcomes in CEAs (quality of life and utility assessment) are difficult to measure in critical illness. Directly ascertaining prior and current quality of life and preferences from patients is challenging, and proxies derived from family members are not necessarily congruent with those of the patient (12). In addition, few studies document long-term survival and quality-adjusted survival after critical illness (13).

Valuing the importance of appropriate end-of-life care is difficult. Death is a frequent outcome of ICU care that is often unavoidable and not necessarily the worst outcome (14). However, current instruments to measure patient preferences generally assume no state worse than death and do not include ways to distinguish between "good" and "bad" deaths.

The burden on family members of critical illness is not easily captured in a CEA. The catastrophic nature of critical illness may be associated with a disproportionate family burden, including emotional distress and time away from other responsibilities, when compared with other conditions (15, 16). Yet, family burden can be difficult to measure and is not fully captured in the PCEHM Reference Case guidelines.

Data on the costs of ICU therapies are often derived from sources with different practice patterns and cost structures. This creates two problems. First, different health care systems are structured differently, in part due to differences in the relative costs of different aspects of care. Specific interventions can change the use of other health care system resources, and the cost of that change can vary depending on the structure of the health care system. Second, measuring costs in a consistent and uniform way across different health care systems and countries is difficult.

Perspective

Both costs and effects depend on the perspective chosen for the economic evaluation. For example, the cost of a particular antibiotic differs for the patient, the pharmacy department, or the insurer. Failure to define and report a specific perspective can threaten internal consistency (costs and effects should be viewed from the same perspective), can prohibit external comparability (results from two CEAs cannot be compared easily if different perspectives were adopted by each), and can impair appropriate consideration in practice. The PCEHM recommended adoption of the societal perspective for the Reference Case because it represents the public interest and, by necessity, considers all potential costs and health effects. Thus, it provides both the broadest perspective and the most comprehensive data, permitting subsequent, more narrowly constructed decisions.

We agreed with the adoption of this perspective but with certain caveats. The societal perspective forces consideration of outcomes and costs not usually considered in critical care studies and a time horizon longer than most critical care studies. Determining costs and outcomes from the societal perspective may be easier for some health care systems than for others. Rather than abandon the societal perspective when studies do not measure appropriate costs or outcomes, we recommend the construction of economic decision models (discussed below) that allow incorporation of data from different sources and of assumptions in the absence of data.

A CEA that fails to provide a societal perspective cannot provide a Reference Case, thus limiting its comparability. However, it may still have value. Many local decisions, such as which pneumonia prevention strategies to use in an ICU, can be informed by a CEA with a more focused institutional perspective. Consider a study, conducted from the hospital perspective, that demonstrated a particular ventilator-associated pneumonia prevention strategy was effective but increased hospital costs. Using a threshold for an acceptable cost per ventilator-associated pneumonia averted, the ICU director could use the study to determine whether she should allocate a part of her budget to this preventive therapy. Because the unit of effect is "ventilator-associated-pneumonia-averted," a limitation is that it is not obvious whether health care dollars might have yielded greater gain in quality-adjusted survival if used in alternative ways. To generate a societal perspective from this study would require an estimation of the consequences of avoiding pneumonia, perhaps derived from studies quantifying the attributable mortality and hospital resource use of pneumonia (17).

Determining Costs

A cost is the use of a resource such that it is no longer available for an alternative use (opportunity cost). The incremental, or additional, costs associated with a given intervention are thus the value of all additional resources used (4). Health care interventions may change costs by affecting several activities, including health care services, patient time expended while undergoing the intervention, indirect activities such as child care or travel while receiving the intervention, subsequent paid and unpaid care, and non-health-related impacts of the intervention. As an extension of adopting the societal perspective, the PCEHM recommended inclusion of each of these costs. Although the relevance of some of these costs to an ICU intervention might seem remote, failure to fully capture the stream of costs incurred by survivors of an intervention could lead to significant underestimation.

Because a CEA is based on incremental costs, the most crucial costs are those that are large and differ significantly between comparison arms. The greater the certainty with which we know these costs, the more confident we can be about the final cost-effectiveness ratio. Practically, this means that, although all costs listed above must be considered, not all must be measured with the same level of detail, and some can be assumed. For example, consider a new anti-endotoxin monoclonal antibody that reduces organ failure and mortality in patients with sepsis. Some important costs that might differ between study arms include the cost of therapy, the costs of ICU care, and the rate at which patients are discharged with residual organ failure (e.g., the number of patients who will require long-term hemodialysis). The cost of therapy can be assumed based on projected pricing from the manufacturer. The costs of ICU care could vary in important and unpredictable ways and ought probably to be measured directly. The costs of hemodialysis can probably be determined as the average cost of hemodialysis, perhaps derived from the literature, multiplied by the projected number of person-years during which hemodialysis is required.

Assigning a monetary value to a service or consumed resource is not trivial. A discussion of approaches to measuring non-health care and postdischarge costs can be found elsewhere (5). There are three general approaches to measuring acute health care costs, ranging in complexity from detailed resource tracking and cost assignment to the use of adjusters, or ratios, with which we multiply health care "prices" (usually referred to as charges) to estimate the actual costs.

Detailed resource tracking and cost assignment systems, such as the TSI system (Transitions Systems Inc., Boston, MA), track all resources used and assign estimates of cost to each resource, usually based on the direct acquisition costs for supplies and time-motion studies for labor costs. Other, less detailed direct measures of resource use include the therapeutic index severity score (TISS) (18), activity-based costing (19), and weighted length of stay (20), but these measures require cost calibration (i.e., a dollar value must be assigned to each TISS point or to each day).

The second approach is to use payment schedules, the most common example being the Medicare fee schedule. This method can be complicated by several factors, including the distorted (noncompetitive) health care market, failure to include copayments and deductibles, and failure to account for variation in costs by geographic region, health care organizational structure, or patient and population characteristics (21).

The third and most common approach multiplies a hospital's charges for a patient's care by a hospital or department-specific ratio of costs to charges (RCC). In the United States, these ratios are available for every hospital (and departments within each hospital) that participates in the Medicare program. RCCs have been shown to be inaccurate for individual patients but accurate to within 10% for groups of patients (22). Physician costs are generally estimated separately from hospital costs in the United States and included elsewhere. Health care systems outside the United States have generally invested less effort in tracking the resources consumed per patient and there is no international standard for detailed costing of health care services.

A further layer of complexity is that the costs for activities can include fixed and variable components. For example, the cost of a computed tomography (CT) scan includes both the variable costs of consumed supplies and personnel time and the fixed costs that are some amortized portion of the initial acquisition cost of the CT scanner. The extent to which a cost is fixed is dependent on the time horizon, and costs can also be semifixed. Furthermore, an activity may have direct and indirect costs. For example, the cost of performing an ICU procedure includes direct costs, such as personnel time, and indirect costs, such as some portion of the administrative costs required to run the ICU. Direct, variable costs are most easily traced to a new therapy whereas indirect and fixed costs may be harder to track. A costing system that fails to distinguish between these components may therefore be inaccurate. However, the work involved in determining the relative contributions to cost of each component is considerable, there is debate over the optimal methodology, and it is often most feasible to make certain assumptions, the impact of which can then be tested in the sensitivity analysis.

Determining Outcomes

Although many potential outcomes can be assessed in critical care (23) (e.g., 30-day survival or organ failure-free days), the outcome recommended by the PCEHM is quality-adjusted survival (measured as the net gain in QALYs) (4). The QALY is a measure of health outcome that assigns to each period of time a weight ranging from 0 to 1, corresponding to the quality of life during that period, where a weight of 1 corresponds to perfect health and a weight of 0 corresponds to a health state judged equivalent to death. The number of QALYs represents the number of healthy years of life that are valued equivalently to the actual outcome (5). This numeric value for a given health state is a utility. Utilities can be measured either by comparing one health outcome state with another (e.g., standard gamble [5] or time trade-off [24, 25]) or by administering a questionnaire for which standard utility weights have been determined (e.g., the Quality of Well-being Scale [26, 27], EuroQOL [28], or the Health Utilities Index [29]).

The advantage of using QALYs is that they combine number of years saved as well as the quality of those years. Thus, treatments with similar survival but different qualities of life will be shown to be different. The use of QALYs in CEAs allows one to compare the relative costs and benefits of widely divergent treatments, such as human anti-endotoxin antibody for the treatment of gram negative sepsis (30), hemodialysis in the ICU (10), or lung transplantation (31). A major disadvantage of using QALYs is that they are relatively difficult to measure, and individual responses may be dependent on the method used for assessment. Furthermore, the underlying assumption (individuals would trade years of life in a given health state for fewer years in perfect health at a constant rate) is not necessarily true.

For QALYs to be useful in CEAs, the utilities for different health states produced by the intervention under study should be measured systematically in a large number of subjects. As an extension of the societal perspective, the PCEHM recommends use of the general public's utilities, rather than those of patients (5). However, the general public often has a worse perception of the quality of life with illness than patients themselves (32, 33). Thus, the cost-effectiveness of an ICU intervention that only returns a patient to her baseline health is underestimated by the societal perspective in comparison with the perspective of those living with chronic illness or disability. Similarly, the general public may view time spent in the ICU as being of poorer quality than that perceived by ICU patients (34, 35). It is not easy, however, to determine the utilities of patients at risk for, and having suffered through, a critical illness. Although there has been increasing research examining the quality of life of patients before and after an ICU stay (34, 36), a number of methodological problems exist (13). Estimating QALYs for time in the ICU, especially for those subgroups of patients with prolonged ICU stays, is also problematic. Nevertheless, some of these problems are no more problematic than for other fields, and we would encourage investigators to measure quality-adjusted survival in future ICU studies. When such data become available, we further recommend that CEAs of critical care interventions consider using patient, as well as societal, preferences. It will be particularly important to review analyses that provide different results when using different perspectives.

An additional challenge for CEAs of critical care is the appropriate evaluation of death. The utilities used for QALY analysis identify a utility of zero as equivalent to death but do not acknowledge that patients can reliably identify health states that are worse than death (14, 41). Arguably, ICU interventions may be likely to produce such health states. Furthermore, by valuing death at zero, we cannot easily differentiate between a "good" death and a "bad" death, confounding our ability to assess the value of interventions designed to improve end-of-life care. To illustrate, if we presume no differences in survival, then there is no gain in utility with the application of therapies or programs that improve care of the dying patient. If we assign a value for improved quality over the few remaining days in the ICU due to improved care, we may measure a gain in utilities but the effect will be tiny (an increment of only a few quality-adjusted days, rather than years). This seems counterintuitive given the general desires of both clinicians and the general public to have good end-of-life care (42).

Comparator

The PCEHM suggested that the comparator group represent the least costly standard practice. Although we agreed with this recommendation, we recognized that defining such a state is often difficult in intensive care. Many ICU therapies are used in heterogeneous patient groups, with evidence for their effectiveness either lacking or confined to only a small subset of patients. Thus, the standard arm may represent current practice, which itself may not even be effective, let alone cost-effective. A new therapy may be cost-effective with respect to current practice but cost-ineffective in comparison with best practice guidelines. Arguably, this problem exists for other areas of medicine but may be worse for critical care given its heterogeneous patient populations.

We recommend that, where possible, comparison groups represent current best practice. In general, one might define best practice as that which consists of adoption of management strategies based on the best available evidence. For example, a study of a new therapy for patients with acute respiratory distress syndrome (ARDS) should ensure that patients are ventilated with low tidal volumes as per the recent findings of the ARDS Network (46). If best practice is considered different from average practice (e.g., if low tidal volumes are not considered to be widely adopted in clinical practice), or if there is more than one predominant model of existing care, we recommend that the new therapy be compared with more than one comparison group. This may be approached either through prospective clinical trial design or sensitivity analysis.

Time Horizon

The observation period over which a CEA is performed, referred to as a "time horizon," must be long enough to capture the important clinical and economic consequences of the therapy. Generally speaking, the time horizon can be as short as 1 year for one-time interventions or as long the patient's lifetime for screening effects of chronic disease. The PCEHM favored a time horizon long enough to capture all relevant future effects. Currently, most efficacy trials in critical care do not monitor patients beyond 30 days or hospital discharge. This duration is obviously short for accurate estimation of the effect on QALYs. Many ICU patients may still be hospitalized at 30 days, and there are many reports of poor quality of life and increased risk of death in ICU survivors many months or even years after ICU discharge. We therefore recommend that studies looking at the effectiveness of a new treatment have longer follow-up. Even if the primary clinical endpoint is mortality at 30 days, monitoring patients to determine survival, costs of care, and quality of life for 6 months or more would greatly enhance our ability to estimate the cost-effectiveness of ICU therapies.

We also endorse the PCEHM recommendation to discount future costs and effects at an annual rate of 3% in the reference case. Discounting recognizes that immediate gains are worth more than future gains ($5 is worth more now than $5 in 1 year) and discounting both costs and effects avoids favoring interventions that occur later (the Keeler-Cretin procrastination paradox) (47).

Constructing the Cost-effectiveness Model and Quantifying Uncertainty

The estimates necessary to determine a cost-effectiveness ratio are derived from different sources and associated with different degrees of uncertainty and precision. To integrate these estimates and test how their uncertainty affects the final result, we construct a mathematical model (48). Unlike experimental designs, which report on specific data from a given set of patients in a specific situation, mathematical models report the expected outcomes of hypothetical cohorts of patients with specific given characteristics. Thus, they are abstractions of reality that do not fully represent a complete picture of a clinical situation but a simplified construct that makes explicit important relationships. Although similar to statistical models, such as APACHE II (49), that produce estimates of expected survival, cost-effectiveness models produce simultaneous estimates of alternative costs and benefits depending on a particular choice of therapies.

There are several types of mathematical models, varying in complexity and clinical realism. Simple models can be analyzed by computerized spreadsheets, by calculator-based analysis, or by hand. For example, suppose that the appropriate triage and initiation of antibiotics in the emergency room decrease the number of elderly patients with pneumonia who are admitted to the ICU. A simple model might estimate the cost savings to be approximately equal to the product of the number of patients who avoided ICU stays and the average cost of an ICU admission for pneumonia. A more complex analysis can be produced by the use of a decision model, a methodology that explicitly describes the events that may occur under each of several strategies being compared. The basic methodology is to draw the assumed structure of the problem (what can happen if strategy A or B is chosen) and calculate the expected value of the outcome, whether it is expected mortality, quality of life, or costs.

Simple decision trees are limited in their ability to describe events that occur multiple times in the care of a patient (e.g., reintubations or ICU readmissions) or interventions that alter the probability of a future event. If clinical realism demands that these characteristics such as recurrent events be included in the model, more sophisticated methodologies are available that incorporate time directly. The most common modeling technique is called a Markov process, a mathematical representation of recurrent events that divides the process being modeled into a series of states that represent the distinct clinical characteristics (50). Modeling techniques, such as discrete events simulation, can also allow the investigator to incorporate physical constraints, such as the number of ICU beds, and arbitrarily complex characteristics, such as the spatial distribution of ICUs in a region. For further detail on these models, the reader is referred elsewhere (5, 51).

Sensitivity analyses. Sensitivity analyses are performed by systematically varying a parameter (or group of parameters) in the model and observing the effects of that variation on the outcome of interest. Any number of parameters can be varied simultaneously. Even where the confidence limits on a particular parameter are large (the parameter is not accurately known), sensitivity analysis can indicate whether the model produces clinically meaningful differences when evaluated between the confidence limits. This ability to perform multiple "what if" scenarios can lead to substantial understanding of the underlying relationships between parameters and provide information on which parameters are most critical. An example of the output from a sensitivity analysis is provided in Figure 1.

View larger version (23K): [in this window] [in a new window]   Figure 1.   Sensitivity analysis from a cost-effectiveness model. The diagram represents possible output from a sensitivity analysis of a cost-effectiveness model of a particular intervention. Denoted a "cost-effectiveness space," any new intervention can be more, less, or equally effective and more, less, or equally costly in comparison with standard, or current, therapy. A standard reference to a specific cost-effectiveness ratio (e.g., $50,000 per life-year saved) can be drawn (cost-effectiveness ratio reference). Given baseline assumptions, the new therapy is compared with current therapy and a specific cost-effectiveness ratio is estimated (base case analysis). Repetitive evaluations of the model can be constructed with different estimates of the various parameters that are used as inputs (costs, effectiveness, mortality, etc). For example, the probability of 30-day survival could be varied higher and lower from its baseline estimate, as could the cost of a particular therapy. If confidence limits surrounding the individual parameters are known, confidence boundaries around the cost-effectiveness ratio can also be constructed. This allows for the construction of estimates of the likelihood of the various potential cost- effectiveness ratios of the intervention. Area A represents the likelihood that the intervention is more cost effective than the (arbitrary) reference standard, area B represents the likelihood that the cost-effectiveness ratio of the intervention lies above the standard threshold, and area C represents the likelihood that the new therapy is both cheaper and better than the current therapy.

Model validity. Although models are extremely powerful tools, able to explore the effects of multiple parameters, one consequence of their flexibility is that determining the validity of their results is more complex. Specifically, where the significance of results from a randomized trial is often expressed by a p value, a model is not so restricted. Proponents for modeling point out that, despite their widespread adoption, p values themselves have important limitations. They indicate only the chance that a study result, drug is better than placebo, could have been random. In contrast, models can investigate more practical questions such as "at what level of risk does therapy A become superior to therapy B?" However, the confidence with which one interprets the model's conclusions comes not from statistical tests on the model's outputs but arises from the validity of the model's structure and assumptions and the hypothesized relationship between outputs and inputs (52, 53).

Reporting

We agreed with the PCEHM recommendation that, when reporting a CEA, authors include a PCEHM Reference Case. The Reference Case is the results ascertained under the PCEHM recommendations for each methodological aspect described above and in Table 2. However, we anticipate that, in CEAs of ICU interventions, the reference case may frequently rely heavily on modeling and assumptions. Where CEAs are conducted in conjunction with randomized trials, we therefore also recommend that authors provide a "data-rich" case, based on as few assumptions as possible. For example, if only 28-day mortality is available from the trial, we would expect an analysis of the costs per life saved at day 28 (data-rich case) in addition to an estimate of the costs per life time quality-adjusted survival (PCEHM Reference Case). We still recommend, however, that this data-rich case be conducted from the societal perspective to facilitate comparisons across studies. These Reference Cases do not preclude additional cases being presented at the author's discretion.

We also agreed with the PCEHM recommendation that authors should provide both a journal report and full technical report (available on request or at a Web site), where greater detail regarding modeling assumptions and data sources can be provided. Finally, we recommended that investigators conducting CEAs in parallel with randomized trials should distribute their proposed analysis plan (perhaps also at a Web site) before unblinding data to minimize bias (54).

Implications

CEA and social utilitarianism. Ultimately, the goal of CEAs is to provide decision-makers with information that will be used to choose between medical care options when all options cannot be purchased. When funds are limitless, the key question is "what is the best?" and CEAs are not needed. When funds are limited, the key question is "what is the best value?" The CEA holds out the potential to address some of the challenging allocation dilemmas facing modern medicine. These decisions are difficult because they involve disagreements about values; and individuals may differ in how they rank these values. There are many ways to distribute goods in a society that emphasize different values (Table 3). The CEA does not provide a value-free method for resolving these disputes. Instead, it applies a specific set of values from which the options can be assessed, using the same perspective. To this, it adds a quantitative result from an explicit, hopefully reproducible, methodology.

Uses of the CEA. The CEA is useful even if it is not used explicitly to prioritize health care expenditures. The process of developing accurate CEAs forces investigators to consider the evidence supporting effectiveness of the treatment, develop a decision tree that encompasses all of the risks and benefits of the therapy, and assess the costs of the intervention. CEAs can also provide unexpected information about treatments that is not ethically problematic. Some apparently expensive treatments (e.g., the use of special care beds to prevent decubitus ulcers in selected ICU patients [55]) save money in the long run ("dominant" treatments, represented by the lower right quadrant of Figure 1). CEAs can also identify treatments that are worse than their alternatives and more expensive ("waste" treatment, the upper left quadrant in Figure 1).

Values competing with utility. In practice, a variety of forces may conspire to make limiting therapy on the basis of CEAs difficult (Table 4). These may occur at the policy level, at the institutional level, and at the individual clinician-patient level. Imagine a decision by Medicare to withhold dialysis for elderly patients with multiple organ failure in the ICU in favor of covering all prescription medication for Medicare patients. Even a clinician who agreed with these cost-effectiveness data and understood the benefits of prioritizing the health care dollar would find herself in the situation of no longer offering a beneficial treatment to a patient who would, perhaps just a week before under different allocation rules, have received it. Clinicians and the hospital might justifiably be worried about the malpractice implications of withholding a treatment on the basis of CEAs if that therapy is effective and is the standard of care in the community. Indeed, a variety of variables such as age, race, substance abuse, sex, and compliance may all be useful in optimizing the cost-effectiveness of a therapy but are problematic in allocating health care resources (56). Allocating funds on the basis of formal CEAs has foundered in the United States on just these concerns (57, 58).

Rescue and false rescue. Perhaps the most frequently raised concern about CEAs in critical care is the "rule of rescue" (58). When the lives are specifically identifiable, their risk of death is imminent, and we have a therapy that may help avoid death, there is a perceived imperative to act. The psychology of rescue decision-making and the inherent value it assigns to identifiable lives in danger inherently violates the assumptions of utilitarianism. Human decision-making is frequently influenced by irrational biases in thinking (59), and rescue decision-making may be one example of a cognitive bias.

Summary

CEAs will play an increasingly important role in the evaluation of ICU interventions. Although we have some concerns regarding the application of the PCEHM guidelines to critical care, we generally concur with their recommendations regarding methodology and reporting. We encourage critical care researchers to conduct reference case analyses in future CEAs. We also encourage new work into the assessment of patient utilities and quality of life before, during, and after critical illness. In addition, we encourage further efforts to develop practical, accurate, and sensitive measures of cost and resource use. We recommend greater use of mathematical modeling to quantify uncertainty in CEAs and to determine the factors to which CE ratios are most sensitive. Finally, we encourage critical care journals to promote clear and transparent reporting of CEAs to facilitate their critical appraisal and consideration by readers. Although we do not expect resource allocation to be based solely on CEAs, we believe that more frequent, more rigorous CEAs of ICU interventions will allow more informative health care policy regarding the care of the critically ill.

Appendix 1: Members, Second American Thoracic Society Workshop on Outcomes Research

Chairs. Derek C. Angus, MD, MPH, University of Pittsburgh, Pittsburgh, PA; Gordon D. Rubenfeld, MD, MSc, University of Washington, Seattle, WA; and Michael R. Pinsky, MD, University of Pittsburgh, Pittsburgh, PA.

Subgroups: Determining costs. I. Larry Cohen, MD, State University of New York, Buffalo, NY; Alfred F. Connors, Jr., MD (session chair), University of Virginia, Charlottesville, VA; David L. Edbrooke, MD, Royal Hallamshire Hospital, Sheffield, UK; Philip Jacobs, PhD, University of Alberta, Edmonton, Alberta, Canada; James Lambrinos, PhD, Union College, Schenectady, NY; Dinis Miranda, MD, Academisch Ziekenhuis, Groningen, The Netherlands; and Clare Hibbert, BA (Hons), Royal Hallamshire Hospital, Sheffield, UK.

Determining outcomes. Deborah J. Cook, MD, MSc, McMaster University, Hamilton, Ontario, Canada; J. Randall Curtis, MD, MPH (session chair), University of Washington, Seattle, WA; Walter T. Linde-Zwirble, Health Process Management Inc., Baltimore, MD; Rui Moreno, MD, Hospital de St. Antonio dos Capuchos, Lisbon, Portugal; and Joel Tsevat, MD, MPH, University of Cincinnati, Cincinnati, OH.

Modeling in economic analyses. Donald B. Chalfin, MD, MS, Beth Israel Medical School, New York, NY; Mark Hlatky, MD, Stanford University, Palo Alto, CA; Mark S. Kamlet, PhD, Carnegie Mellon University, Pittsburgh, PA; Mark S. Roberts, MD, MPP (session chair), University of Pittsburgh, Pittsburgh, PA; and Frank A. Sonnenberg, MD, UMDNJ/RWJ Medical School, New Brunswick, NJ.

Implications of economic analyses. David Asch, MD, University of Pennsylvania, Philadelphia, PA; E. Haavi Morreim, PhD, University of Tennessee, Memphis, TN; Judith R. Lave, PhD, University of Pittsburgh, Pittsburgh, PA; Michael Rie, MD, University of Kentucky, Louisville, KY; and Gordon D. Rubenfeld, MD, MSc (session chair), University of Washington, Seattle, WA.

Appendix 2: Glossary

Annual discount rateAn annual discount applied to adjust for the time-value of money. It reflects the fact that costs (and effects) are worth less in the future than they are today. To illustrate, a bank lending a dollar today expects that, to be "repaid in full" at some future time, it requires both the original dollar (capital) plus an additional sum (interest) that reflects the duration of time the bank was without the original dollar. Although discounting is widely accepted for costs, it is more controversial when applied to health effects.

Cost-effectiveness ratioA ratio of the difference in costs divided by the difference in health effects between two competing interventions. In other words, the ratio expresses the incremental cost of obtaining a unit of health effect (e.g., life-year) for a given health intervention when compared with an alternative.

CostsUsually expressed in monetary terms, they are a measure of what we forfeit to achieve a utility or acquire an item.

Efficacy versus effectivenessEfficacy describes the clinical effects under ideal circumstances (usually a controlled clinical trial). Effectiveness describes the clinical effects under typical "real world" circumstances, where patients are not carefully selected and practice is not carefully monitored.

PCEHMThe U.S. Public Health Service Panel on Cost-effectiveness in Health and Medicine. This panel published guidelines and recommendations for the conduct and reporting of cost-effectiveness analyses with the goal of improving and standardizing health economic studies.

PCEHM Reference CaseThe Reference Case is the calculation of a cost-effectiveness ratio, using a standardized methodology, thereby facilitating comparison of ratios for different technologies across studies.

PerspectiveThe point of view from which the analysis occurs. Defining the perspective clarifies who incurs costs and effects. For example, costs borne by the hospital are incorporated in an analysis conducted from the hospital's perspective but are not incorporated in an analysis conducted from the patient's perspective.

QALYQuality-adjusted life-year. The QALY is a measure of health outcomes, known as a utility, that incorporates both the duration and quality of survival. Quality of life must be expressed as a numeric value ranging from 0 to 1, and the duration of time that a patient exists in that state is adjusted by the quality of life of being in that state. For example, if the quality of life for a particular condition is 0.5, and the patient remains in that state for 1 year, 0.5 QALY has accrued. The quality of survival must be measured with an instrument that provides a numeric value (ranging from 0 to 1). Such instruments are known as utility instruments and include the standard gamble, the time trade-off, and certain health status surveys.

Sensitivity analysesAnalyses that recalculate cost-effectiveness ratios while varying assumptions in the equation to determine the sensitivity of the ratio to a particular assumption. Multiway sensitivity analyses imply more than one assumption is varied simultaneously.

Standard gambleA method to determine utility. A subject is asked to compare the certainty of continuing life in a particular health state against a gamble. The gamble has two outcomes, typically perfect health (1) or death (0). The probabilities of the gamble resulting in 1 or 0 are systematically varied until the subject is indifferent about the choice of continued life or the gamble. At this point, the gamble probability is equal to the subject's preference for her health state. For example, if the gamble probability of 70% chance of perfect health (and 30% chance of death) is considered equivalent to continued life in the current health state, that health state has a value of 0.7.

Time horizonThe time over which a study estimates costs and effects (or utilities). In a cost-effectiveness analysis, the time horizon should extend far enough into the future to capture the major health and economic outcomes both intended effects and unintended side effects. Frequently, the appropriate time horizon extends beyond the availability of primary data, and modeled data must be used in the analysis.

Time trade-offA method to determine utility. Here, the subject is asked to determine what amount of time in a better health state is equivalent to a longer length of time in a poorer health state. For example, a subject is given a life expectancy in their current state of 10 years and asked to compare that with a life expectancy with perfect health. If the subject equates 7 years in perfect health as equivalent to 10 years in their current health, then the utility she assigns to her current health is 0.7 (7/10 years).

UtilitarianismA theory of social justice that holds that policies that produce the greatest good for the greatest number improve social welfare. This theory incorporates everybody's well-being into the social policy by balancing the utility of those who gain with the utility of those who lose from a given policy.