INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Despite widespread utilization of pulmonary artery catheters (PACs) to aid in management of critically ill patients, few prospective studies have been performed to validate the clinical effectiveness of this diagnostic tool. One case-control study demonstrated that patients managed with PACs experienced higher morbidity and mortality than similarly ill patients who were managed without PACs (1). Failure of clinicians to interpret PAC data accurately could have contributed to these findings. Several studies have demonstrated that both physicians (2) and nurses (3) frequently misinterpret the pulmonary artery occlusion pressure presented in questionnaire format. However, no previous study has assessed intra- and interobserver variabilities of interpretation of PAC data or compared nurse and physician readings in clinical practice. In this study, we hypothesized that clinically significant intra- and interobserver variabilities occur in the interpretation of pulmonary artery occlusion pressures amongst physicians and nurses who care for critically ill patients.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Our investigational review board waived formal review of this protocol. This study was conducted between June 1997 and March 1998 in our medical and cardiac critical care units. All critical care nurses (CCNs) who participated in this study had undergone extensive initial training in interpretation of PAC tracings as per nursing department policies and had already been interpreting PAC tracings at the bedside prior to the start of this study. These skills are reevaluated by nursing clinical educators on an annual basis. About 25% of our 50 nurses are certified by the American Association of Critical Care Nurses and the remainder have undergone extensive in-house critical care training in addition to their registered nurse training.

During the study period, nurses were asked to obtain contiguous pulmonary artery and pulmonary artery occlusion pressure (Ppao) paper tracings twice a day for each patient with a PAC, at a time that they recorded a value for the Ppao on the chart. The standard practice for our nurses is to interpret the Ppao using the on-screen cursor; paper tracings are rarely evaluated by nursing staff. The Marquette (Marquette Electronics, Milwaukee, WI) monitors used in this study allow users to adjust an on-screen horizontal axis (the "cursor") tangential to the point on the Ppao tracing believed to be end-exhalation. A digital Ppao reading is displayed for that horizontal axis, which is the value typically recorded by our bedside nurses on patient charts. Paper tracings were recorded on the Marquette Centralscope 12C Central Station (Marquette Electronics) at the same time that nurses recorded values, and tracings were later presented to our Chiefs of Critical Care (CCMD) and Cardiology (CARD), who interpreted them in blinded fashion on two separate occasions.

To validate the correlation of cursor and paper measurements, one investigator (C.A.M.) measured the Ppao using the cursor and collected simultaneous paper tracings. Thirty measurements were performed in 12 patients, at intervals of > 3 h, for no more than 5 measurements per patient. This same observer later interpreted these 30 paper tracings in a blinded fashion. These data were not included in analyses of observer variabilities.

For the main study of observer variabilities in Ppao interpretation, a single investigator (T.A.) indicated the mode of ventilation on each tracing. He also assessed each tracing for the presence or absence of ventricular waves, defined as phasic, > 4 mm Hg increases in Ppao corresponding to the ST interval of the electrocardiogram, and measured the respiratory phasic variation (RPV) of all Ppao tracings, defined as the maximum-minimum value on each tracing.

Our reviewers also suggested that we obtain a wider measure of internurse and interphysician variability in Ppao interpretation. After completing analysis of observer variabilities, we chose three paper tracings; one of a ventilated patient in whom there was little or no RPV and where CCMD, CARD, and CCN agreed, one of a ventilated patient with both large RPV and interobserver differences, and one of a spontaneously breathing patient with both large RPV and interobserver differences. These tracings were labeled as to ventilator status and then interpreted by a convenient sample of CCNs and physicians who frequently use PACs.

Statistical Analyses

Values of Ppao recorded by nurses and the first set of readings measured post hoc by physicians were compared for interobserver differences. The degree of inter- and intraobserver agreements was indexed with linear correlation coefficients. Because clinical decisions are based on whether the Ppao readings are low, normal, or high, the individual readings were categorized as < 5 mm Hg for low, 5-15 mm Hg for normal, and > 15 mm Hg for high. The levels of agreement among the observers for these categories were then indexed with both unweighted and weighted (4, 5)  statistics. The differences in the readings between observers (CCMD-CARD, CARD-CCN, and CCMD- CCN) were also plotted against the mean of the respective readings (the Bland-Altman method) (6) to ascertain the pattern and spread, and to evaluate any systematic differences, in readings. Limits of agreement, defined as the interval expected to contain 95% of the interobserver differences, were constructed for each comparative set. The limits of agreement are given by the mean difference plus or minus 2 (or, more accurately, 1.96) standard deviations. Ninety-five percent confidence intervals were constructed around the limits of agreement to evaluate their precision and consistency across the comparative groups. The effects of the presence of positive pressure ventilation, ventricular waves, and RPV on observer variabilities were assessed using stratified and multiple linear regression analyses. A p value of < 0.05 signified significance.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Correlation of Cursor and Printed Measurements of Ppao

Thirty pulmonary artery paper tracings were compared with values determined by synchronous cursor measurement, in 12 patients. All but four measurements were performed in mechanically ventilated patients. The cursor method of Ppao measurement was found to correlate well with measurement on paper tracings (r = 0.93, see Figure 1).

View larger version (13K): [in this window] [in a new window]   Figure 1.   Correlation of Ppao values measured simultaneously: on-screen cursor versus paper tracings.

Analyses of Observer Interpretations of Ppao

One hundred and forty-seven measurements of Ppao were performed on 40 patients with a mean age (± SE) of 62.5 ± 2.2 yr and a mean APACHE II (acute physiology and chronic health evaluation system II) score of 21.5 ± 0.8. Either or both physician readers found 28 tracings as not containing an appropriate Ppao tracing, and 6 tracings were excluded from analyses because they were mislabeled as to ventilation status on initial interpretation (see DISCUSSION). Seventeen of 113 tracings were obtained in patients who were spontaneously breathing and the remainder were obtained from patients during positive pressure ventilation. Mean resolution on the tracings was 4.1 ± 0.1 mm Hg/mm.

Intraobserver variability in Ppao interpretation. Intraobserver correlation coefficients for measurements of Ppao were 0.91 for the CCMD and 0.87 for the CARD. Intraobserver variability was independently associated with a greater degree of RPV of Ppao for CCMD observations and with ventilator status for CARD observations. Correlation coefficients for CCMD and CARD for tracings with < 8 mm Hg RPV were 0.95 and 0.92, respectively; and 0.88 and 0.82 for tracings with RPV  8 mm Hg. Intraobserver differences were significantly greater for nonventilated patients and in tracings with RPV  8 mm Hg as compared with RPV < 8 mm Hg (1.8 versus 0.9 mm Hg, p < 0.05) for CARD readings. Intraobservational differences tended to be greater in tracings with RPV  8 mm Hg as compared with RPV < 8 mm Hg (1.5 versus 0.8 mm Hg, p = 0.06) for CCMD readings.

Interobserver variability in Ppao interpretation. Interobserver correlation coefficients ranged from 0.66 for CARD- CCN comparisons to 0.83 for CCMD-CARD comparisons (all p < 0.05). For the small subcohort of tracings in nonventilated patients (n = 17), interobserver correlations ranged from 0.69 to 0.76. Similarly, for tracings of ventilated patients (n = 96), the correlations ranged from 0.60 to 0.84.

Figure 2 demonstrates the degree of interobserver variability as assessed by the Bland-Altman method. There were clinically significant differences between the readings of all observers. The range of differences varied from 11 to 12 mm Hg for CCMD-CARD to 13 to 15 mm Hg for CARD-CCN. The limits of agreement ranged from 5.3 to 7.3 mm Hg for CCMD-CARD comparisons, from 7.6 to 8.0 mm Hg for CCMD-CCN, and from 9.2 to 7.6 mm Hg for CARD-CCN. The range and 95% confidence intervals of the limits of agreement of physician-nurse differences were consistent with those for physician-physician differences. Interobserver variability was not explained by the scale of the tracings, the presence of v waves, or positive pressure ventilation. The magnitudes of interobserver variabilities were independently associated with RPV for all three group comparisons (p < 0.05). The absolute values of interobserver differences in tracings with RPV  8 mm Hg (mean for the population) were significantly greater than for tracings with variations < 8 mm Hg (p < 0.05, except p = 0.10 for CCMD-CCN). Similarly, respiratory phasic variations in Ppao tracings tended to be or were significantly greater in outliers than in strips where interobserver differences were within the limits of agreement (p values of 0.03, 0.05, and 0.11).

View larger version (19K): [in this window] [in a new window]   View larger version (19K): [in this window] [in a new window]   View larger version (18K): [in this window] [in a new window]   Figure 2.   Interobserver comparisons of Ppao measurements. Mean values of Ppao for the compared observers are plotted on the abscissa, and differences are plotted on the ordinate. (A) Differences between physician observers; (B) differences between cardiology chief and critical care nurses; (C ) differences between critical care chief and nurses. Ninety-five percent confidence intervals are also highlighted.

When Ppao readings were categorized as low (< 5 mm Hg), normal (5-15), and high (> 15), unweighted  values were 0.57 for CARD-CCMD, 0.51 for CARD-CCN, and 0.41 for CCMD-CCN comparisons. Both unweighted and weighted  statistics are presented in Table 1. On the basis of CCMD readings, 34 (30%) of 113 CCN Ppao values could have led to different therapeutic decisions (either failure to treat or inappropriate treatment compared with CCMD) at these clinical thresholds.

We presented the tracings to a convenient sample of 23 nurses from our medical intensive care unit and 18 physicians (8 certified cardiologists, 4 certified pulmonary critical care physicians, and 6 cardiology and pulmonary fellows). All but one nurse and one physician (trainee) identified the Ppao within 4 mm Hg of the mean value (13 mm Hg, range 6-22 mm Hg, SD 2.6 mm Hg) for the tracing with minimal RPV (see Figure 3A). For the tracing with large RPV in a ventilated patient, group variability was twice that observed for the tracing (A) with minimal RPV (mean 13 mm Hg, range 5-24 mm Hg; SD 5.2 mm Hg, see Figure 3B). Variability of readings for the tracing with large RPV in a spontaneously breathing patient was midway between tracings A and B (mean 19 mm Hg, range 10-23 mm Hg, SD 3.3 mm Hg; see Figure 3C). The magnitudes of variabilities of Ppao interpretation were similar for nurses and physicians for all three tracings.

View larger version (48K): [in this window] [in a new window]   View larger version (48K): [in this window] [in a new window]   View larger version (46K): [in this window] [in a new window]   Figure 3.   Interobserver variability in Ppao interpretation: nurses (n = 23) versus physicians (n = 18). The patient in (A) was receiving ventilator assistance and had little or no RPV. The patient in (B) was receiving ventilator assistance, had large RPVs and there was significant variability in interpretation of this tracing (both CARD- and CCMD-CCN outlier). The patient in (C ) was breathing spontaneously, had large RPVs, and there was also significant variability in interpretation of this tracing (both CARD- and CCMD-CCN outlier). Note that the Ppao precedes the pulmonary artery pressure tracing in (B) and that tracing (C ) is only of the Ppao. Note also that the distributions of Ppao values were not greater for critical care nurses than for physicians who routinely use PACs. Ninety-five percent confidence intervals for the distributions of Ppao are also illustrated. Squares indicate certified CCN; circles, CCN (not certified); triangles, pulmonologist; inverted triangles, cardiologist; diamonds, fellow.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

This study demonstrates the presence of clinically significant differences in the measurement of pulmonary artery occlusion pressure tracings among physicians and between physicians and critical care nurses. Differences between the physicians and nurses for 95% of the readings (the limits of agreement) ranged from 5.3 to 7.3 mm Hg (CCMD-CARD) to 9.2 to 7.6 mm Hg (CARD-CCN). Neither the use of positive pressure ventilation nor the presence of ventricular waves in Ppao tracings predicted physician-nurse or physician-physician disagreement in measurement. Tracings with larger respiratory phasic variations were associated with significant intra- and interobserver differences in measurement of Ppao across most compared groups.

The major reason for interobserver differences in Ppao interpretation appears to be related to different methods of identification of end-exhalation. For instance, the CARD identified end-expiration on the basis of ventilator status of the patients; for those receiving ventilator assistance, he usually chose end-exhalation as the trough values of Ppao; for those not receiving ventilator assistance, he chose peak values of Ppao. In contrast, the CCMD determined end-exhalation by combining knowledge of the patients' ventilator status and examination of the tracings for respiratory phasic cycle lengths, assuming that exhalation is usually longer than inhalation. In tracings for ventilated patients with large RPVs, he frequently chose the peaks for Ppao interpretation, if the peaks were of longer duration than the troughs. Thus differences in Ppao interpretation appear to arise, at least in part, owing to differences in how clinicians choose to identify end-exhalation on tracings. The relatively high degree of intraobserver variability suggests a lack of consistency in the application of each observer's criterion and/or objective difficulty in identifying end-expiration irrespective of the criteria used.

This study shows, for the first time, that the degree of respiratory phasic variation of intrathoracic pressures impacts observer variability of readings to a similar degree in nurses and physicians. Because disagreement in identifying end-exhalation on tracings appears to contribute to increased interobserver variability, large RPVs are expected to cause larger interobserver variability. In cases of extreme respiratory phasic variation of intrathoracic pressures, even acknowledged experts appear to be unsure as to appropriate methods of measuring Ppao. For example, Leatherman and Marini (7) suggest that in cases of large respiratory fluctuations in which esophageal manometry is not available, "the mean Ppw should probably be recorded. . . ." A previous study by Hoyt and Leatherman (8) demonstrated that in ventilated patients with large RPVs, the difference in Ppao before and after muscle relaxation was related to the magnitude of RPV. Our study confirms this finding, extends the concept to nonventilated patients, and suggests that future educational programs designed to improve performance of critical care personnel should emphasize appropriate measurement of pressure tracings with significant respiratory phasic variability.

Previous studies (2, 3) have examined interpretation of a "clearly reproduced" tracing presented in a questionnaire format. In these studies only 53% of physicians and 58% of critical care nurses were able to identify correctly the Ppao within 5 mm Hg of its true value. Our study differed in that 147 tracings were interpreted and there was no definitive "gold standard." Although it is possible that our findings could merely define a local deficiency in Ppao interpretation, 95% of our physicians and nurses were able to identify the Ppao within 4 mm Hg (resolution of the strip) on a tracing with minimal RPV (see Figure 3A), which is substantially better performance than was noted on the questionnaires cited above. Thus our physicians and nurses are unlikely to be much worse (or better) at identifying an accurate Ppao than are similarly qualified personel elsewhere. Accordingly, we believe that our results are generalizable to other institutions.

To our knowledge, this is the first study that demonstrates the fidelity of the on-screen cursor method of Ppao measurement used by nurses at many institutions. Although our data directly apply only to our (Marquette) instruments, cursor measurements correlated well with those measured on paper (correlations were comparable to intraobserver variabilities of paper tracing interpretation). Cursor measurements may even be more accurate because resolution on paper can range from 3-6 mm Hg/mm, whereas the cursor provides a precise digital whole number display of the horizontal tangent chosen by the user to represent Ppao. On paper tracings, the observer must first choose end-exhalation correctly, and when it falls in between hatch marks, must estimate within 1-3 mm Hg.

The primary limitation of this study is the lack of a reference standard (the true Ppao) for comparison. Thus absolute values of accuracy could not be computed for any of the observers. Also, six tracings were not included in this analysis because they were mislabeled regarding patient ventilator status. Exclusion of these tracings from the analysis did not change any observed relationships. Finally, one could argue that being present to observe patient respirations during Ppao measurement could have caused systematic bias (in favor of nurses) for comparisons with physician readings. However, physician intra- and interobserver differences (for CCMD-CARD and for the larger convenience sample of nurses and physicians) were also large and cannot be explained by failure to observe the patients.

The differences between critical care nurse readings and those of our two chief physicians were not substantially different; the limits of agreement as illustrated in the Bland-Altman curves were similar. In fact, we were surprised that the differences were not greater because nursing measurements were performed by a large, heterogeneous group with variable proficiencies in measuring Ppao. Moreover, our observations of Ppao interpretations by 18 physicians versus 23 CCNs suggest that a similar degree of variability permeates both physician and nurse observations (see Figure 3). Nonetheless, the observed differences between physician and nurse interpretations of Ppao have potentially significant clinical implications. In our hospital, most measurements are performed by bedside nurses; the majority of medical therapies depend on the dependability of these measurements. We can compute the number of times interpretations between nurses and our physicians differed sufficiently to potentiate therapies (or no therapies) that the physicians might have acted differently on had they been present to read the measurements themselves. For the sake of argument, if one considers a Ppao < 5 mm Hg as grounds for treating a given patient with fluids and a Ppao > 15 mm Hg as grounds for treating with diuretic, then nursing measurements could have led to treatments that would not have been intended by our critical care chief in 30% of readings. Moreover, nurses recorded Ppao values on 28 (19%) occasions in which our physicians felt that the tracing was not valid for assessing the Ppao (usually felt to be "overwedging" or failure to obtain a true "wedge" tracing). Because the theorized utility of PAC measures is to inform appropriate management, failure to interpret the data correctly could contribute to inappropriate therapies in a substantial proportion of critically ill patients.

In 1996, Connors and the Support Study Group (1) published a retrospective cohort study comparing the outcomes of critically ill patients who had versus those who did not have a PAC during their hospitalization. This study included more than 5,700 patients, who were assessed for numerous demographic and physiologic variables so as to select and then rigorously match the two groups for severity of illness. Patients whose care included a PAC experienced a significantly higher 30-d mortality, longer ICU lengths of stay, and higher costs. Several hypotheses have been generated to explain these results. First, despite the attempts to assure matching, the patients in the PAC group may have been more ill. Another explanation is that the benefit obtained from PACs, in this heterogeneous group of patients, from different hospitals, was outweighed by the inherent risks of PACs (e.g. infections, pneumothoraces) (9). Alternatively, some of the information obtained from the PACs could have been incorrect and/ or clinicians could have used accurate data to prescribe inappropriate therapies. The degree of variability in our study suggests that PAC data are frequently interpreted incorrectly and, although not addressed by our study, such errors are likely to potentiate suboptimal therapies. Insofar as critically ill patients have less physiologic reserve to tolerate such errors, it is plausible that PACs could contribute to worse outcomes through improper use. Significant efforts are currently under way to create educational programs to improve interpretation of PAC data. Our data highlight that such efforts should be aimed at both physicians and bedside nurses who regularly work with PACs.

In conclusion, this study confirms the findings from previous studies (2, 3) in describing significant interobserver variability in measurement of PAC pressure tracings. Nurse-physician differences were not substantially greater than physician- physician differences. Significant differences in measurement were more common in patients whose Ppao had a higher degree of respiratory phasic variation, which likely resulted in disagreement in identifying end-expiration on the tracings. The observed differences in Ppao measurement are likely of sufficient magnitude as to be clinically significant and may lead to inappropriate clinical decisions and interventions with potentially harmful consequences.