Published ahead of print on July 15, 2004, doi:10.1164/rccm.200402-147OC
© 2004 American Thoracic Society doi: 10.1164/rccm.200402-147OC Idiopathic Interstitial PneumoniaWhat Is the Effect of a Multidisciplinary Approach to Diagnosis?Division of Pulmonary and Critical Care Medicine and Department of Radiology, University of Michigan Health System, and Department of Biostatistics, University of Michigan School of Public Health, Ann Arbor, Michigan; University of California, San Francisco, San Francisco, California; University of Washington, Seattle, Washington; Mayo Clinic, Scottsdale, Arizona; and Armed Forces Institute of Pathology, Washington, DC Correspondence and requests for reprints should be addressed to Fernando J. Martinez, M.D., M.S., 3916 Taubman Center, 1500 E. Medical Center Drive, Ann Arbor, MI 48109-0360. E-mail: fmartine{at}umich.edu
Current guidelines recommend that the clinician, radiologist, and pathologist work together to establish a diagnosis of idiopathic interstitial pneumonia. Three clinicians, two radiologists, and two pathologists reviewed 58 consecutive cases of suspected idiopathic interstitial pneumonia. Each participant was provided information in a sequential manner and was asked to record their diagnostic impression and level of confidence at each step. Interobserver agreement improved from the beginning to the end of the review. After the presentation of histopathologic information, radiologists changed their diagnostic impression more often than did clinicians. In general, as more information was provided the confidence level for a given diagnosis improved, and the diagnoses rendered with a high level of confidence were more likely congruent with the final pathologic consensus diagnosis. The final consensus pathologist diagnosis was idiopathic pulmonary fibrosis in 30 cases. Clinicians identified 75% and radiologists identified 48% of these cases before presentation of the histopathologic information. Histopathologic information has the greatest impact on the final diagnosis, especially when the initial clinical/radiographic diagnosis is not idiopathic pulmonary fibrosis. We conclude that dynamic interactions between clinicians, radiologists, and pathologists improve interobserver agreement and diagnostic confidence.
Key Words: diagnosis • nonspecific interstitial pneumonia • usual interstitial pneumonia Histopathologic subsets of idiopathic interstitial pneumonia exhibit different prognoses (1–8). Therefore, an accurate diagnosis is critical to the management of patients with idiopathic interstitial pneumonia. Clinical features, high-resolution computed tomography (HRCT) (9–12), and surgical lung biopsy (13) all play a role in establishing a diagnosis. An American Thoracic Society/European Respiratory Society (ATS/ERS) committee emphasized the need for a dynamic diagnostic integrated process in which clinicians, radiologists, and pathologists exchange information in the determination of a diagnosis in individual patients with suspected idiopathic interstitial pneumonia (14). Using histology alone as the "gold standard" for diagnosis can be complicated by difficulties with interrater agreement (15) and the potential for sampling error (2). The combination of HRCT and histologic features also better predicts prognosis compared with either modality alone (16). In the current study, we hypothesized that an interactive process would improve the interobserver agreement between expert clinicians, radiologists, and pathologists in consecutive patients with suspected idiopathic interstitial pneumonia compared with each group working in diagnostic isolation. This study illustrates that a consensus diagnosis, reached after a careful exchange of clinical, radiographic, and histopathologic information, often differs from the initial diagnosis reached by the individual clinician, radiologist, or pathologist working in isolation. Some of these results have been previously reported in the form of an abstract (17, 18).
Patient Selection Patients in this study represented consecutive patients referred to the University of Michigan Specialized Center of Research (SCOR) in the Pathobiology of Fibrotic Lung Disease for potential participation in research protocols between January 2002 and August 2002. Patients with suspected idiopathic interstitial pneumonia were referred to the study center by participants in the University of Michigan Fibrotic Lung Disease Network (see ACKNOWLEDGMENT). Through the course of evaluation all patients underwent a history, physical examination, complete pulmonary function testing, HRCT, and surgical lung biopsy. Patients without an HRCT scan or a surgical lung biopsy were excluded. Patients with known collagen vascular disease at the time of presentation were excluded. Patients without collagen vascular disease at initial presentation, but that developed a discrete collagen vascular disease during the course of follow-up, were included.
Data Collection
Pathology Interpretation
Study Organizational Scheme
Statistical Analysis Each observer's diagnosis was coded into one of six categories—idiopathic pulmonary fibrosis (IPF), nonspecific interstitial pneumonia (NSIP), respiratory bronchiolitis interstitial lung disease/desquamative interstitial pneumonia (RBILD/DIP), hypersensitivity pneumonitis, bronchiolar disease, and other. McNemar tests were subsequently used to test whether two probabilities of agreement conducted during different steps or by different raters were equal. A  statistic allowing for multiple raters was also used to assess agreement in diagnosis. Scores are rated as almost perfect agreement (above 0.8), substantial agreement (scores between 0.6 and 0.8), moderate agreement (scores between 0.4 and 0.6), fair agreement (scores between 0.2 and 0.4), slight agreement (scores between 0.0 and 0.2), and poor agreement (scores below 0.0) (19). An estimating equation approach to the analysis of correlated statistics was used in comparisons of statistics estimated throughout the study and in producing confidence intervals for the statistics (20). A  2 or Fisher exact test was used to examine the relationship between the confidence of each participant for their diagnosis and the final pathologist consensus diagnosis. A recorded confidence level of 1 was considered high in confidence and a confidence level of 2, 3, or 4 was considered low in confidence.
Fifty-eight patients were evaluated. The for the two pathologists without clinical or radiologic information (pathology review before Step 1) was 0.72 (95% confidence interval, 0.57 to 0.86) with a probability of agreement of 0.81 (95% confidence interval, 0.71 to 0.91). IPF and NSIP were the most common diagnoses (Tables 1 and 2). In general, agreement among clinicians and between clinicians and radiologists improved when more data (clinical, radiographic, and pathologic) were provided (Table 3). Importantly, when pathology results were provided to the group, the radiologists were more likely to alter their interpretation than were the clinicians, as enumerated by the decrease in intraobserver coefficient between Steps 3 and 4 (Table 4).
The agreement level of clinicians and radiologists with the pathologist consensus diagnosis tended to increase as additional information was provided (Table 5). Importantly, a diagnosis of IPF by each clinician or each radiologist following the review of clinical and radiographic data (before pathologic data) coincided with the pathologic diagnosis in all but two cases. For one case two clinicians and both radiologists changed their diagnosis to NSIP; in a separate case one clinician changed to a diagnosis of desquamative interstitial pneumonia. Furthermore, clinicians correctly diagnosed more cases of IPF that were subsequently pathologically confirmed as IPF compared with radiologists at this step (Step 3) of the study. After pathologic impressions were provided to the group, the radiologists changed their interpretations to a greater extent. This is suggested by the marked improvement in interobserver agreement between radiologists and pathologists (Table 5) and the lower radiologist intraobserver agreement level (Table 4). The radiologists most often changed their diagnosis of NSIP to IPF and changed diagnoses of hypersensitivity pneumonitis and RBILD/DIP to NSIP (data not shown). The pathologists changed their initial consensus impressions (n = 11) less often than the clinicians (n = 27, 17, 15) or radiologists (n = 34, 28). These data reflect the change in diagnosis for the pathologists between their initial consensus review before obtaining clinical and radiographic information; for clinicians and radiologists the change reflects a change from each individual's diagnosis after a clinical and radiologic discussion (Step 3) but before histopathologic discussion (Step 5). When the pathologists changed their initial consensus impressions, their final diagnoses are listed in Table 6. It is notable that in three cases a histologic diagnosis of UIP was thought not to represent IPF after review of clinical and HRCT data. We were not able to identify unique clinical or radiographic features that could reliably predict non-IPF idiopathic interstitial pneumonia diagnoses without a surgical lung biopsy.
Clinicians tended to be more confident than radiologists in the early steps of the evaluation process, although the number of confident diagnoses increased for all observers as more information was provided (Table 7). A statistically significant relationship between confidence in diagnosis and final diagnosis was more often seen with clinicians compared with radiologists, particularly in the early compared with late stages of the evaluation process. An increase in confidence was noted with the addition of pathologic data for both clinicians and radiologists.
Total agreement (3 clinicians, 2 radiologists, and 2 pathologists) on a final diagnosis was reached in 47 (81%) of the cases (IPF, n = 28; NSIP, n = 13; RBILD, n = 1; hypersensitivity pneumonitis, n = 1; other, n = 4). In 6 (10%) cases all but one rater was in agreement (IPF, n = 2; NSIP, n = 2; hypersensitivity pneumonitis, n = 2). In four cases all but two raters were in agreement (RBILD, n = 1; bronchiolar, n = 3). In one case the raters were equally divided between RBILD versus a bronchiolar disorder.
Assigning a diagnosis to a patient with interstitial lung disease is difficult and at times imprecise. In clinical practice, pulmonary physicians, radiologists, and pathologists are separated by time, geographic location, and different schedules. This challenges the pulmonary physician confronted with the actual patient in the clinic to individually integrate and analyze the gathered clinical, radiographic, and pathologic data and make decisions about a given patient rather than discussing the data with the radiologist and/or pathologist in a collective manner. In an attempt to improve diagnostic accuracy the ATS/ERS consensus statement recommended a dynamic integrated approach to the diagnosis of interstitial lung diseases that involves an interaction between clinicians, radiologists, and pathologists to achieve a final clinical, radiologic, and pathologic diagnosis (14). In this study we examined whether such a dynamic interactive process impacted the perceived diagnosis, and confidence level for that diagnosis, for a group of consecutive patients with suspected idiopathic interstitial pneumonia. We demonstrate that:
This study quantifies how a dynamic exchange of clinical, radiographic, and pathologic information changes the diagnostic impression of expert physicians when assigning a diagnosis of idiopathic interstitial pneumonia. Exchanges of information lead to both changes in diagnosis and improved confidence with the assigned diagnosis. The effect of interaction between physicians seems most critical for non-IPF idiopathic interstitial pneumonia cases. After review of the clinical, radiographic, and pathologic information the pathologists thought that IPF was the diagnosis in 30 cases. Of these 30 cases, the two radiologists and three clinicians each identified 14, 15, 22, 23, and 23 (respectively) after the review of clinical/radiographic information before obtaining histopathologic information. Importantly, in almost half of the overall cases the histopathologic findings were critical in determining the final diagnosis from each participant. The discussion and exchange of clinical, radiographic, and histopathologic information also tended to improve each participant's confidence in their diagnosis. This is important as diagnoses at any stage rendered with a high level of confidence tended to correspond with the final consensus diagnosis. This suggests that when a physician expresses uncertainty about an idiopathic interstitial pneumonia diagnosis, extra effort should be made to seek additional information and/or opinions with the hope of achieving greater confidence in their diagnostic impression. Furthermore, these data strongly support the role of surgical lung biopsy in patients with a non-IPF HRCT and clinical scenario (1, 14). A unique feature of this study is the lack of the arbitrary assignment of a gold standard for the final diagnosis. By allowing all participants the opportunity to change their initial diagnostic interpretations we monitored how diagnoses changed with more information and the level of agreement at each stage in the evaluation process. Our data demonstrate that a change in the final clinical–radiographic–pathologic diagnosis is particularly likely to occur in patients with a clinical and radiographic scenario suggestive of non-IPF idiopathic interstitial pneumonia. Importantly, providing the clinical and radiographic information to pathologists led to an alternative or clarified pathologic opinion in 19% of the cases. These findings extend previous work that evaluated agreement between clinicians, radiologists, and pathologists (5, 10, 16, 21–24). The level of agreement between participants was good at onset and improved with discussion of additional clinical and histopathologic information. The highest initial agreement was between radiologists. Radiologists were also less likely than clinicians to believe that the clinical scenario was characteristic of IPF and were more likely, compared with clinicians, to change their diagnosis on the basis of histopathologic findings. Previous work demonstrated the limited sensitivity (16, 24, 25) yet high positive predictive value (10, 12, 16, 26) of HRCT for detecting IPF. Our data highlight the value of clinicians reviewing the actual HRCT with a radiologist versus relying on reported findings. The greatest improvement for the clinician's interobserver agreement occurred after the discussion of clinical and radiographic features with radiologists. The ATS/ERS consensus panel on idiopathic pulmonary fibrosis states that in the absence of a surgical lung biopsy, the diagnosis of IPF/UIP remains uncertain. However, the panel proposed diagnostic criteria that could be used by clinicians to increase the likelihood of a correct clinical diagnosis of IPF (Table 8). In the immunocompetent adult, the presence of all of the major diagnostic criteria along with at least three of the four minor criteria was recommended. These criteria were not prospectively tested. However, the clinicians involved in the present study were coauthors of the Statement and followed these principles in defining cases of idiopathic interstitial pneumonia. This may account for some of the differences seen between the clinicians and radiologists involved in this study.
A limitation of our study is the use of participants with significant expertise in the field of interstitial lung disease. Some investigators have suggested that expert radiologists exhibit improved interobserver agreement compared with radiologists with lesser expertise (27–30). No similar data have examined this topic in the evaluation of suspected idiopathic interstitial pneumonia by less experienced clinicians. Additional data are required to assess the role of a dynamic, interactive diagnostic process in the hands of less experienced clinicians, radiologists, and pathologists. These data also do not address how individual personalities may have impacted the degree of agreement (or disagreement) at various steps throughout the experiment. The participants in this study are recognized, senior experts and are accustomed to rendering an "expert opinion." As such, other physicians may render diagnostic opinions that are more (or less) malleable compared with this study group. In summary, our data quantify the essence of a dynamic clinical, radiographic, and pathologic approach to the diagnosis of patients with suspected idiopathic interstitial pneumonia. We believe that at a minimum the evaluation of patients with idiopathic interstitial pneumonia should include an interaction between the clinician and thoracic radiologist. If this initial step results in a confident diagnosis of IPF a surgical lung biopsy is not required. In all other cases a surgical lung biopsy is likely to impact the final diagnosis and should be performed if possible. If a surgical lung biopsy is available, the pathologist should be involved in the diagnostic analytic discussion. It appears the current gold standard for the diagnosis of idiopathic interstitial pneumonia is a dynamic integrated process that requires direct interaction between clinicians and radiologists as well as pathologists when a surgical lung biopsy is available. Further research to clarify this process, examine its effect in the hands of less experienced physicians, and evaluate the implications in terms of response to therapy and prognosis is required.
The University of Michigan Fibrotic Lung Disease Network includes: University of Michigan, Division of Pulmonary and Critical Care, Ann Arbor, MI—D. Arenberg, W. Bria, D. Dahlgren, C. Grum, V. Lama, T. Ojo, M. Peters-Golden, R. Simon, T. Sisson, T. Standiford, V. Thannickal, E. White; Internal Medicine Clinic, Alpena, MI—P. Bachwich, C. Easton, J. Mazur; The Lung Center, Battle Creek, MI—S. Chaparala, G. Harrington, N. Potempa; Bay City, MI—S. Manawar, J. Summer; Clawson, MI—P. Hukku, J. Sung; Clinton Township, MI—R. Babcock; Pulmonary and Critical Care Medicine Consultants, Commerce, MI—J. Belen, M. Dunn, D. Maxwell, R. Reagle, R. Sherman, S. Simecek; Oakwood Hospital, Dearborn, MI—L. Victor; Henry Ford Hospital, Detroit, MI—B. DiGiovine, M. Eichenhorn, R. Hyzy, J. Popovich, Jr., D. Spizarny; Botsford General Hospital, Farmington Hills, MI—B. Rabinowitz; Pulmonary and Critical Care Specialists, Farmington Hills, MI—G. Ferguson, P. Kaplan, S. Sklar, W. VanderRoest; Pulmonary Associates, PC, Flint, MI—O. Filos, V. Rao, M. V. Thomas, J. Varghese, J. Vyskocil, F. Wadenstorer; Grand Valley Internal Medicine, Grand Rapids, MI—J. Cantor, W. Katz, R. Johnson, Jr., D. Listello, J. Wilt; Michigan Medical Professional Company, Grand Rapids, MI—C. Acharya, W. Couwenhoven, T. Daum, M. Harrison, M. Koets, G. Sandman, G. VanOtteren; Michigan Medical, PC, Holland, MI—S. Kraker; Huntington Woods, MI—M. Greenberger, A. O'Neill, D. Wu; Pulmonary Clinics of Southern Michigan, Jackson, MI—R. C. Albertson, III, J. Chauncey, T. Murray, G. Patten; Associated Pulmonary and Critical Care Specialists, PC, Kalamazoo, MI—T. Abraham, J. Dirks, B. Dykstra, G. Grambau, J. Schoell; Pulmonary and Critical Care Associates, PC, Kalamazoo, MI—R. Brush, S. Jefferson, J. Miller, S. Schuldheisz, M. Warlick; Pulmonary and Critical Care Consultants, Lansing, MI—J. Armstrong, A. Atkinson, T. Kantra, L. Rawsthorne, D. Young; Pulmonary Services, Lansing, MI—A. Abbasi, C. M. Gera, G. Kashyap, J. Morlock; Respiratory Medicine, Marquette, MI—S. Danek, A. Saari; Midland, MI—S. Yadam; Central Michigan Healthcare System, Mt. Pleasant, MI—E. Obeid; Muskegon Pulmonary Associates, Muskegon, MI—D. Hoch, A. Kleaveland; Owosso Medical Group, Owosso, MI—A. Allam, M. A. Gad, Jr.; Lung Associates, Pontiac, MI—A. Desai, U. Dhanjal, A. Sethi; St. Joseph's Hospital, Pontiac, MI—F. Ahmad, L. Kaiser, L. Rosenthal, D. Sak; Physician HealthCare Network, Port Huron, MI—R. Ailani, M. Basha, A. Hadar, S. Holstine; Pulmonary, Critical Care, and Sleep, PC, Rochester Hills, MI—M. W. Al-Ameri, R. Go, M. Kashlan; Rochester, MI—K. Aggarwal; Roseville, MI—W. Hanna, R. Marchese; William Beaumont Hospital, Royal Oak, MI—R. Begle, D. Erb, K. P. Ravikrishnan, J. Seidman, S. Sherman; Saginaw, MI—R. Agarwal, F. Ansari, T. Damuth, C. Indira; Spring Lake, MI—M. Ivey; Lakeside Healthcare Specialists, St. Joseph, MI—S. Deskins, A. Palmer, S. Shastri; Pulmonary and Critical Care Associates, St. Clair Shores, MI and Troy, MI—R. DiLisio, S. Galens, K. Grady, D. Harrington, R. Herbert, C. Hughes, J. Lee, A. Starrico, K. Stevens, M. Trunsky, W. Ventimiglia; Taylor, MI—D. Mahajan. Pulmonary Medicine Associates, Warren, MI—H. Kaplan, L. Tankanow; Henry Ford Wyandotte Hospital, Wyandotte, MI—M. Pensler; Toledo Pulmonary and Sleep Specialists, Toledo, OH—F. O. Horton, III, A. Nathanson, R. Wainz.
Supported in part by National Institutes of Health NHLBI grant P50HL46487, NIH/NCRR 3 MO1 RR00042-33S3, NIH/NIA P60 AG08808-06, NHLBI, 1 K24 HL04212, and 1 K23 HL68713. Conflict of Interest Statement: K.R.F. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; T.E.K. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; G.R. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; J.P.L. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; T.V.C. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; W.D.T. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; B.H.G. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; E.A.K. received $3,000 in 2003 from General Electric (GE) for speaking at a conference and received grant support from GE Medical Systems, August 1, 2002–November 1, 2002 ($37,733) and February 20, 2003–February 16, 2005 ($44,000) and serves on the General Electric Radiology Research Academic Fellowship (GERRAF) Board of Directors and has received $3,000 annually since 2002 and no compensation from the GE Medical Advisory Board; G.B.T. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; Q.L. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; S.M. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; V.N.L. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; S.E.G. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript; F.J.M. does not have a financial relationship with a commercial entity that has an interest in the subject of this manuscript. Received in original form February 3, 2004; accepted in final form July 11, 2004
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
TOP
ABSTRACT
METHODS