Training Pulmonary and Critical Care Physicians in Outcomes Research
Should We Take the Challenge?

J. RANDALL CURTIS, GORDON D. RUBENFELD, and LEONARD D. HUDSON

Division of Pulmonary and Critical Care Medicine, Department of Medicine, University of Washington, Seattle, Washington

	INTRODUCTION

TOP INTRODUCTION SUMMARY REFERENCES

Pulmonary and critical care training programs have been training physicians to perform medical research for 40 years. Early in the history of these training programs, the majority of these trainees developed their skills in physiologic research. During the last 20 years, increasing numbers of trainees have turned their attention to cell and molecular biology. More recently, a rapidly expanding number of trainees have become interested in pursuing clinical epidemiology or health services research. This trend has followed the increasing emphasis of investigators, health care delivery systems, research funding agencies, and government on outcomes research. Our goal is to define outcomes research, describe why it is important that some pulmonary and critical care physicians be trained as outcomes researchers, and discuss some of the important components of developing an outcomes research training program for pulmonary and critical care physicians.

	WHAT IS OUTCOMES RESEARCH?

"Outcomes research" is a term developed in the past 15 years to describe clinical research concerning the outcomes of medical care (1, 2). Outcomes, broadly defined, can include any variable used to assess the effect of some medical treatment or intervention. "Outcomes research," however, connotes clinical research that focuses less on physiology or biochemical processes and more on clinical end points directly relevant to patients and society: mortality, quality of life, health status, functional status, and costs of care (3, 4). Outcomes research examines the effectiveness of treatment in real clinical settings of heterogeneous patient populations with barriers to treatment rather than the efficacy of treatment in the more controlled research settings with carefully selected, motivated subjects. In addition, outcomes research is concerned with a broad range of issues touching on all aspects of health care delivery, from the clinical encounter between patient and clinician to questions of the organization, financing, and regulation of the health care system. While large, population-based randomized controlled studies using outcomes important to patients (often called "effectiveness trials") are a type of outcomes research, much of outcomes research relies on observational, nonrandomized designs (5). In fact, outcomes research performed by trainees will likely be observational research because of the time and expense involved in performing effectiveness randomized controlled trials. There is a growing recognition of the need for outcomes research, both observational and experimental, to understand and improve the way health care is delivered throughout our society.

Although the terms "outcomes research," "clinical epidemiology," and "health services research" have overlapping definitions, we use "outcomes research" in a broad way to include much of the disciplines of clinical epidemiology and health services research. Therefore, we define outcomes research as clinical research attempting to understand and improve the outcomes of illness and medical treatment by focusing on those outcomes important to patients and society.

Although it is difficult to draw a sharp line distinguishing outcomes research from other clinical research, many important types of clinical studies are clearly not outcomes studies. For example, small clinical trials using physiologic or biochemical end points are not outcomes research. Nonetheless, faculty performing these other types of clinical research have much to contribute to training outcomes researchers, and trainees interested in other types of clinical research could benefit from many of the tools taught to outcomes researchers.

Pulmonary clinicians have been doing outcomes research for at least 25 years. As one example, the Nocturnal Oxygen Therapy Trial in the 1970s used quality of life, as well as survival, as an outcome in a large effectiveness trial to demonstrate the value of oxygen therapy for patients with COPD and hypoxemia (6). The investigators used this study to describe the effect of COPD on the quality of life in these patients (7). Since that time, there has been a large body of work assessing the quality of life and functional status of patients with COPD and asthma and using these measures to determine the effectiveness of treatments (8).

Critical care researchers have been among the leaders in assessing outcomes of medical care and performing risk adjustment to control for severity of illness. The APACHE score was one of the earliest validated systems to assess risk, control for severity of illness, and predict outcome (14), and the APACHE III remains one of the most comprehensive and accurate risk adjustment systems available (15). Assessing outcomes and predictors of outcomes in critical care has been an important and vital area of research for three decades.

	WHY SHOULD PULMONARY AND CRITICAL CARE SPECIALISTS BE TRAINED IN OUTCOMES RESEARCH?

It is important that pulmonary and critical care specialists be involved in outcomes research and not leave this field entirely to generalists and Ph.D. researchers. Outcomes research and assessments of quality of care will shape the future of the structure and financing of health care (16, 17). Pulmonary and critical care specialists can sit on the sidelines watching it happen or be active participants designing and conducting research to help shape the future of health care. Future decisions about treatment effectiveness and quality of care will also be based, in part, on outcomes research. To have a role in this research and how it is translated into practice, pulmonary and critical care specialists need rigorous training in its methods and need to contribute to this body of work. There are many key questions about the outcomes of care for individuals with lung disease or for the critically ill that remain unanswered. Pulmonary and critical care specialists have a unique perspective based on their clinical experience and they provide care for patients that use a large proportion of health care resources. These specialists have important insights not only into pulmonary and critical illness but also into the practice patterns of their colleagues. These insights are important for designing and conducting useful outcomes research. Finally, health care delivery systems must make decisions about the most effective and efficient mix of generalists and specialists. Hopefully, these decisions will be based on the goal of providing high quality care within economic constraints rather than simply reducing costs. In order to identify the generalist-specialist mix that provides the highest quality care, we need the data to identify health needs and outcomes of care (18). Pulmonary and critical care physicians should be involved in generating these data, but they need training in outcomes research to do so.

There are also specific clinical areas in pulmonary and critical care medicine that have important unanswered questions that will likely be best addressed with outcomes research. For example, the role of specialist care in asthma, the diagnosis and management of sleep apnea, and the incorporation of patient preferences into decision-making in the intensive care unit are areas in which outcomes research may be able to provide much-needed insight to improve medical care. A study examining the most cost-effective role for specialist in the care of patients with asthma is unlikely to be able to randomize patients to specialist care and will likely be best answered with the observational methods of outcomes research.

As, noted above, pulmonary and critical care specialists have produced a great deal of high quality clinical research that includes important contributions in outcomes research. In the past, these clinical investigators were largely self-trained and often collaborated with epidemiologists and statisticians. So why do we need new training pathways for pulmonary and critical care fellows? The field of outcomes research has become too complex for physicians to train themselves in this research or to perform rigorous outcomes research when their training and interests are in basic science or clinical practice. A career in outcomes research requires the same level of preparation, training, and time commitment as a career in basic research.

During the past two decades, formal training programs have developed to educate physician-investigators in the methods of outcomes research. Many of these training programs have been developed within Divisions of General Internal Medicine (19), and they have often used the resources of Schools of Public Health to teach their trainees the principles of biostatistics, clinical epidemiology, and health services research. The Robert Wood Johnson Clinical Scholars Program has been training physician-investigators in outcomes research for more than 20 years and serves as a model program. If pulmonary and critical care fellowships are interested in training fellows to perform outcomes research, it behooves these programs to invest the time and resources to offer State of the Art training.

	WHAT IS NEEDED TO TRAIN PULMONARY AND CRITICAL CARE FELLOWS IN OUTCOMES RESEARCH?

The training program for these clinical investigators can be divided into three components: training in the methods of clinical investigation; research projects undertaken by the trainee; and an intellectual environment in which to develop the skills of collaborating with other investigators. An outcomes research training program must attend to each of these three components of research training.

1. Training in the Methods of Outcomes Research

Clinical investigators need a background in the core disciplines of biostatistics, epidemiology, and health services research, just as basic science investigators need a background in physiology, biochemistry, and molecular biology. The basic curriculum for an outcomes research training program built on a foundation of clinical epidemiology, biostatistics, and health services research is shown in Table 1. In addition, some clinical research fellows may choose further training in such disciplines as health economics, medical ethics, sociology, and anthropology. Clinical investigators do not need to be statisticians, epidemiologists, or anthropologists, but they need an understanding of these disciplines in order to design and conduct clinical research and collaborate with colleagues in these areas.

The curriculum of these training programs will depend in part on the strengths of a given institution and in part on the interests of the trainee. However, these programs should all provide basic instruction in choosing a research question and selecting a study design to answer the question. Trainees need introductory courses in biostatistics that describe the foundations of statistical inference and the types of statistical tests. Because much of outcomes research is observational and must attempt to control for many confounding factors, trainees also need an in-depth understanding of multivariate modeling, including linear regression, logistic regression, and survival analyses. Trainees should also have an understanding of the complexities of survey design and the value of qualitative research.

The graduate schools affiliated with many university-based training programs can provide this type of broad-based education, and some trainees will opt for a Master's degree in public health, epidemiology, biostatistics, or health economics. A Master's degree provides recognition of a trainee's education and is generally associated with a systematic and thorough training in the disciplines and tools of outcomes research. The Master's degree may also be a significant asset in building a career since potential employers will use this degree as a marker for research training. It is important that the Master's degree program be tailored to the past experiences and training of a physician-fellow; the standard course work and course load designed for the student having just finished a baccalaureate degree is not most appropriate for the physician-fellow. In addition, it is important to emphasize that successful outcomes research training programs can provide similar course work and training without conferring a Master's degree (20). Similarly, although employers use the Master's degree as a marker of training, a history of completing and publishing quality research is more important than a degree. Although some trainees may wish to obtain a Ph.D. in one of these fields, such additional training will generally not be provided in the setting of a pulmonary and critical care fellowship.

2. Trainees' Research Projects

The most important component of a clinical research trainee's education is the trainee's research projects. Fellows should have experience formulating a research question, designing a study to answer the question, completing the study, and writing the results for publication. Choosing an interesting, important, and feasible research question in one of the most important steps of clinical research (21), and fellows can gain experience in this task early in their research training, provided they have guidance. The completion of high quality research is the best education in any type of research training program and the most important component for helping the trainee move from fellowship to a career in outcomes research (22). In order to provide fellows with the appropriate support for these kinds of projects, it is important to have faculty working in outcomes research in pulmonary- or critical-care-related fields.

3. Intellectual Environment

Finally, an outcomes research training program must provide the intellectual environment in which to develop experience collaborating with other investigators and giving and receiving constructive criticism. Such an environment requires a "critical mass" of clinical investigators, usually both faculty and fellows, to provide a network of colleagues. To achieve a critical mass, programs can make use of faculty outside of the Division of Pulmonary and Critical Care Medicine, provided that fellows have easy access to such faculty and at least one mentor within the division to assure the fellow is getting the necessary support. One mechanism to formally provide this intellectual environment is to develop a conference in which fellows and faculty can present works in progress. The main purpose of this type of conference is for the presenter to receive feedback on the research question, project design, and the interpretation and presentation of results. Therefore, these conferences should be relatively small and should provide a comfortable and supportive atmosphere for the presentation of works in progress. In addition, this conference should be an opportunity for fellows to learn to critique their colleagues in a constructive way.

	DOES AN OUTCOMES RESEARCH TRAINING PROGRAM DIFFER FROM A BASIC SCIENCE TRAINING PROGRAM OR A PROGRAM FOR CLINICIAN-TEACHERS?

The basic components of the training program (background training in a specific discipline, completion of research projects, an intellectual and collegial environment) do not distinguish outcomes from basic research training. A common model for training fellows in basic research is for the fellow to choose a mentor and for the mentor to offer the fellow one of a number of projects that stem from the mentor's work. The laboratory setting often provides the training environment; the mentor, colleagues in the laboratory, and laboratory technicians provide much of the training. This "training by laboratory" approach can apply to clinical research as well. In place of the laboratory group, clinical research trainees will benefit from a small committee of mentors and advisors to help the trainees develop the basic skills and research experience. For example, a mentoring committee for a fellow interested in studying quality of life in patients with idiopathic pulmonary fibrosis might include a Ph.D. advisor with expertise in quality of life measures, a clinician-teacher who runs a referral clinic for interstitial lung disease, and a physician-outcomes researcher.

There are, however, some important differences between basic and outcomes research training programs. Pulmonary and critical care medicine fellowship programs have more experience and expertise in training fellows for basic research careers. Some may argue that programs can train fellows to do outcomes research without having a formal program by virtue of the fellow's clinical experience. However, having fellows teach themselves to do outcomes research is neither an effective nor an efficient way to train outcomes researchers. An outcomes research trainee without the support of an established outcomes research program is no more likely to be successful than a basic science trainee trying to perform research without the support of a laboratory research program. Another difference between basic and outcomes research training is that outcomes research training requires more course work in its core disciplines (especially biostatistics and epidemiology) than do most basic science fields. Part of the reason for more background course work in outcomes research training is that most physicians have had less course work in biostatistics or epidemiology than in the basic sciences. Performance of rigorous outcomes research also requires more course work because an individual study depends on the appropriate use of many of the core disciplines: biostatistics, epidemiology, and health services research.

It is also important to emphasize the difference between an outcomes research training program and a clinician-teacher training program; an outcomes research program is not the appropriate pathway to train individuals interested in a career as a clinician-teacher. Although the clinician-teacher should have an understanding of epidemiology and biostatistics in order to critically evaluate the medical literature, many aspects of the outcomes research training are not relevant to the clinician-teacher. Because few academic pulmonary and critical care medicine fellowship programs offer a formal pathway for clinician-teachers, there will be a tendency for these individuals to choose an outcomes research training pathway because it seems closer to clinical medicine than to basic research. However, fellows should realize that performing multivariate analyses and writing survey questions are as different from caring for patients as is running an electrophoresis gel. Because the clinical research pathway is not designed to meet the needs of these trainees, it is likely to disappoint them. In addition, the clinical research pathway will not be as productive in training future researchers if many of its trainees are not interested in research careers.

	WHAT IS THE OUTCOME OF AN OUTCOMES RESEARCH TRAINING PROGRAM?

Fellows should complete their training with an understanding of the methods of outcomes research, experience conducting and completing a research project, and the skills to collaborate with investigators from the different disciplines and to contribute to the intellectual environment of a clinical research program. At the end of the training, fellows interested in pursuing outcomes research in academic medicine should have several research manuscripts in various stages of completion and generally should have defined for themselves either a clinical topic or a research tool upon which they want to build the next phase of their career.

Some training in understanding and evaluating outcomes research is important for all pulmonary and critical care trainees. Practicing physicians should have the basic skills to understand and evaluate outcomes research because this research will form the basis for much of clinical practice. However, this basic understanding of outcomes research does not require a degree in biostatistics or epidemiology and it does not require the performance of outcomes research. Teaching colleagues to critically evaluate outcomes research is another important role for those training in outcomes research training programs.

	SUMMARY

TOP INTRODUCTION SUMMARY REFERENCES

During the last 20 years, physicians practicing medicine, investigators examining effectiveness of medical treatments or evaluating the quality of care, and policy-makers debating health care financing reform have increasingly focused on the outcomes of medical care. Performance of high quality outcomes research that truly informs these individuals requires training and expertise. University-based pulmonary and critical care medicine fellowship programs that have the experience and resources in outcomes research should take the lead in developing postgraduate training pathways to advance this scientific discipline and support outcomes research that will help shape the future practice of pulmonary and critical care medicine. Many university-based pulmonary and critical care fellowship programs have the experience and resources within their institutions. However, in addition to having the resources, training programs must invest the time and energy to make these resources easily accessible to fellows. Therein lies the challenge.

	Footnotes

Correspondence and requests for reprints should be addressed to J. Randall Curtis, M.D., M.P.H., Division of Pulmonary and Critical Care Medicine, Harborview Medical Center, Box 359762, 325 Ninth Avenue, Seattle, WA 98104-2499.

(Received in original form July 9, 1997 and in revised form November 12, 1997).

Acknowledgments: The writers would like to thank Drs. Joan G. Clark, Richard B. Goodman, Thomas R. Martin, and H. Thomas Robertson for their careful reviews of the manuscript and their helpful comments.

	References

TOP INTRODUCTION SUMMARY REFERENCES

1. Roper, W. L., W. Winkenwerder, G. M. Hackbarth, and H. Krakauer. 1988. Effectiveness in health care: an initiative to evaluate and improve medical practice. N. Engl. J. Med. 319: 1197-1202 [Medline].

2. Epstein, R. S., and L. M. Sherwood. 1996. From outcomes research to disease management: a guide for the perplexed. Ann. Intern. Med. 124: 832-837 [Abstract/Free Full Text].

3. Fowler, F. J., P. D. Clearly, J. Magaziner, D. L. Patrick, and K. L. Benjamin. 1994. Methodological issues in measuring patient-reported outcomes: the agenda of the work group on outcomes assessment. Med. Care 32:JS65-JS76.

4. Deyo, R. A.. 1991. The quality of life, research, and care. Ann. Intern. Med. 114: 695-697 .

5. Iezzoni, L. I. 1994. Risk Adjustment for Measuring Health Care Outcomes. Health Administration Press, Ann Arbor, MI.

6. Nocturnal Oxygen Therapy Trial Group. 1980. Continuous or nocturnal oxygen therapy in hypoxemic chronic obstructive lung disease. Ann. Intern. Med. 93: 391-398 .

7. McSweeney, A. J., I. Grant, R. K. Heaton, K. M. Adams, and R. M. Timms. 1982. Life quality of patients with chronic obstructive pulmonary disease. Arch. Intern. Med. 142: 473-478 [Abstract/Free Full Text].

8. Curtis, J. R., R. A. Deyo, and L. D. Hudson. 1994. Health-related quality of life among patients with chronic obstructive pulmonary disease. Thorax 49: 162-170 [Free Full Text].

9. Jones, P. W. 1995. Issues concerning health-related quality of life in COPD. Chest 107(Suppl. 5):187S-193S.

10. Jones, P., F. H. Quirk, C. M. Baveystock, and P. Littlejohns. 1992. A self- complete measure of health status for chronic airflow limitation: The St. Georges Respiratory Questionnaire. Am. Rev. Respir. Dis. 145: 1321-1327 [Medline].

11. Juniper, E. F., G. H. Guyatt, R. S. Epstein, P. J. Ferrie, R. Jaeschke, and T. K. Hiller. 1992. Evaluation of impairment of health-related quality of life in asthma: development of a questionnaire for use in clinical trials. Thorax 47: 76-83 [Abstract/Free Full Text].

12. Juniper, E. F., P. R. Johnston, C. M. Borkhoff, G. H. Guyatt, L. P. Boulet, and A. Hankioja. 1995. Quality of life in asthma clinical trials: comparison of salmeterol and salbutamol. Am. J. Respir. Crit. Care Med. 151: 66-70 [Abstract].

13. Jones, P. W., and the Group at NSQoLS. 1994. Quality of life, symptoms and pulmonary function in asthma: long-term treatment with nedocromil sodium examined in a controlled multicentre trial. Eur. Respir. J. 7: 55-62 [Abstract].

14. Knaus, W. A., J. E. Zimmerman, D. P. Wagner, E. A. Draper, and D. E. Lawrence. 1981. APACHE: acute physiology and chronic health evaluation: a physiologically based classification system. Crit. Care Med. 9: 591-597 [Medline].

15. Knaus, W. A., D. P. Wagner, E. A. Draper, J. E. Zimmerman, M. Bergner, P. G. Bastos, C. A. Sirio, D. J. Murphy, T. Lotring, A. Damino, and F. E. Harrell. 1991. APACHE III prognostic systems: risk prediction of hospital mortality for critical ill hospitalization adults. Chest 100: 1619-1636 [Abstract/Free Full Text].

16. Wennberg, J. E., M. J. Barry, F. J. Fowler, and A. Mulley. 1993. Outcomes research, PORTs, and health care reform. Ann. N.Y. Acad. Sci. 703: 52-62 [Medline].

17. Davis, K.. 1993. Health care reform in the United States: the contribution of health services research to the debate. Ann. N.Y. Acad. Sci. 703: 287-290 [Medline].

18. Nash, D. B., and I. S. Nash. 1997. Building the best team. Ann. Intern. Med. 127: 72-74 [Free Full Text].

19. Goldman, L.. 1991. Blueprint for a research career in general internal medicine. J. Gen. Intern. Med. 6: 341-344 [Medline].

20. Hulley, S. B., S. R. Cummings, W. S. Browner, T. B. Newman, and N. Hearst. 1988. Designing Clinical Research. Williams & Wilkins, Baltimore.

21. Kahn, R. C.. 1994. Picking a research problem: the critical decision. N. Eng. J. Med. 330: 1530-1533 [Free Full Text].

22. Applegate, W. B.. 1990. Career development in academic medicine. Am. J. Med. 88: 263-267 [Medline].