INTRODUCTION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Goiter is a common disorder. The Whickham survey (1) in a nonendemic area of the United Kingdom reported that 16% of the population studied had a goiter. Upper airway obstruction (UAO) is one of the complications from a goiter, and in an earlier study (2) we found that 31% of subjects with a clinically diagnosed symptomatic goiter had UAO. Although this might be an overestimate, it suggested that there was a large and probably undiagnosed disease burden. It is very important to recognize whether a goiter is producing UAO since the treatment is likely to be of a surgical nature, which is different from the treatment of nonobstructing goiter and from other causes of airway obstruction.

The diagnosis is an important one to make, but clearly recognizable changes in lung function caused by UAO tend to become apparent only when the cross-sectional area of the airway has been reduced by more than 50% (3). A high index of suspicion is needed, with a recent study showing that symptoms of airway obstruction were present in only 42% of patients with UAO caused by goiter (4), and breathlessness was reported in only 28%. It has been suggested that all patients with a retrosternal goiter, regardless of symptoms, should have respiratory function tests performed to investigate the possible presence of UAO. Inspection of the maximal expiratory and inspiratory flow-volume curve is currently the simplest method to establish the presence of UAO (2).

We previously found (2) that the most sensitive and specific method for diagnosing UAO in patients with a goiter was the interpretation of a patient's flow-volume loop by an expert. Other investigations, which included plain radiographs and ultrasound, had a significantly lower ability to detect goiters that were impairing ventilatory function. However, relying on a human expert for the diagnosis poses problems concerning the reliability and availability of this expertise. Because the treatment for UAO is likely to involve surgery, with a definite morbidity and risk, the ability to diagnose this condition correctly is crucial. Using more complex tests of lung function such as airway resistance may not be easy to implement since most patients attending a thyroid clinic will need to be tested, and a method that can be easily undertaken within the clinic would be preferable. We have therefore looked at whether a neural network could help identify upper airway obstruction using a series of standard and novel indices calculated from the flow-volume loop, with a view to identifying the best numerical descriptors for identifying UAO and so obviate the need for using a human expert.

	METHODS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Patients and Recordings

All patients attending a specialist thyroid clinic who had a clinical diagnosis of goiter were asked to perform maximal forced expiratory and inspiratory maneuvers. These maneuvers were recorded as a continuous event using an unheated Fleisch pneumotachograph (60 mm internal diameter), the signal from which was sampled at 250 Hz and numerically integrated with respect to time on a computer to give expired volume. The performance characteristics of the system have been previously described (7) and comply with published recommendations (8, 9).

The same technician supervised all the maneuvers, with each patient being asked to perform at least three maneuvers conforming to ARTP/BTS recommendations (9). All the flow-volume loops were stored on computer disk and were subsequently inspected by an observer (MRM) to determine the presence of UAO. Using all the clinical and physiologic data for the 155 patients tested in this way there were 46 patients in whom UAO was diagnosed and these formed Set 1. Loops from 51 subjects with goiter who were thought not to have evidence of UAO were chosen to form Set 2. A third set of 50 loops was chosen from the routinely recorded data of patients with chronic obstructive pulmonary disease (COPD) whose flow volume loops were thought to be consistent with this diagnosis and had no evidence of UAO. A single flow-volume loop from each subject in these three sets of patients was used for subsequent analysis, and the loop chosen was that with the highest peak expiratory flow (PEF) for that subject. Representative loops from the normal, COPD, and UAO sets are shown in Figure 1.

View larger version (13K): [in this window] [in a new window]   Figure 1.   Representative flow-volume curves: normal (top), COPD (middle), and mild UAO (bottom).

The PEF, FEV₁, FVC, and FEV₁/FVC ratio (FEV₁%) were all calculated for each MEFV curve, together with the FEV₁/PEF ratio (FEV₁ in ml divided by PEF in L/min), which has been proposed as a helpful index for detecting UAO (10). For those indices with satisfactory prediction equations (11) the deviation from the predicted value was expressed as standardized residuals to remove age, height, and sex bias, which percent of predicted does not do for data from adults (11, 12). Standardized residuals (SR) are given by:

(1)

where RSD is the residual standard deviation for the regression equation used. An SR of 1.645 means the value lies on the lower 90% confidence limit.

Plots of the curves from all three sets of patients were randomly mixed and coded. The repeatability of expert opinion on the flow-volume loops was then assessed by two independent observers (MRM and PB) who on two separate occasions, more than 8 wk apart and 6 mo after the initial split into the three sets, separated the plots into those with UAO and those without UAO, with the latter including both patients with goiter and patients with COPD. The two observers used only the flow-volume curves in their assessment and were asked to comment on the difficulty in assessing the plots using a scale of 1: not at all certain, 2: moderately certain, 3: quite certain, and 4: very certain. When reexamining the plots the observers were unaware of the classification awarded on the first occasion. Intraobserver and interobserver repeatability were calculated using Cohen's kappa statistic (13) where the difference between the proportion of cases agreed between two observers and that expected by chance is standardized by 1 minus the proportion expected by chance. A value equal to 1 indicates complete agreement.

Neural Networks and Their Inputs

Artificial neural networks are computational models of the activity of the brain that can be used to help solve complex problems of classification. Multivariate statistical analysis can unravel complex relationships between data elements where the operator defines the type of association between the variables, for example, linear, logarithmic, and so on. Neural networks can operate without these constraints to determine a successful strategy that categorizes data by learning from training data sets. From given input variables a network iteratively operates a weighting system that demonstrates which variables contribute most to the categorization of the data. This may then give a greater insight into how the original training data set had been categorized and allow better categorization in the future.

The categorization of data by a neural network first requires the extraction from the data of all possible key features to be considered, and these are then used to undertake the classification process. The neural network consists of groups of nodes (also termed neurons) arranged in layers that communicate with each other either to deliver the categorization or to inform other nodes about their progress toward the categorization. The network starts with an arbitrary weighting applied to the key features extracted from the data, which are used as input variables, and then trains itself by adjusting these weights until its categorization of the training set of data gives the best match with a supplied "correct" categorization. The trained network can then use its previous experience to deliver a categorization on data that it has not previously been exposed to. The method is very dependent on the validity of the supplied "correct" categorization. Because there is no "absolute" standard for the diagnosis of UAO, we have used the principal standard available, that of human expert opinion, based on all the available data, whose expertise has been previously checked by the results achieved from surgical therapy (2), in order to define the categorization of the training data.

In order to extract suitable features from the data for the neural networks to use as input variables, we derived three groups of additional novel indices from the flow-volume data. In subjects with UAO the flow volume curve is relatively flat over the early part of the flow-volume curve (2, 3, 5), and so several types of score for this flatness across the peak of the flow volume curve were derived in the following way. The FVC was divided into 20 equally spaced volume increments. At each 5% increment in FVC the instantaneous flow was taken, as shown in Figure 2, and then expressed as a fraction of the predicted peak flow (FV5, FV10, . . .). Volume chords (VC) were interpolated by computer horizontally across the flow-volume curve at various cut points down from the PEF. In the first instance the cut points were derived at 0.5-L/s decrements in flow below the peak, to a maximum of 2.5 L/s below recorded PEF, as shown in Figure 3. The length of these chords (which are in units of volume) was expressed as a proportion of the recorded FVC (VCA5, VCA10, etc.) and also as a proportion of predicted FVC (VCA5p, VCA10p, etc.). An additional set of chords was derived with cut point decrements equal to 5% of the recorded PEF, and the length of these was expressed as described above (VCP5, etc., and VCP5p, etc.). A final set of chords was derived at point decrements equal to 5% of the predicted PEF, and the chord lengths then only expressed as a proportion of predicted FVC (VCPP5p, etc.).

View larger version (31K): [in this window] [in a new window]   Figure 2.   A plot of an expiratory flow-volume curve of a subject with upper airway obstruction with dotted lines drawn at 5% increments of FVC that intersect the curve at the points plotted with closed circles. The flows at these points are then expressed as percent of predicted PEF to derive FV5, FV10, etc.

View larger version (13K): [in this window] [in a new window]   Figure 3.   A plot of a normal expiratory flow-volume curve with chords drawn at decrements from PEF at 5% of the recorded PEF. Five chords were also calculated at decrements of 5% of the predicted PEF and for decrements of 0.5 L/s.

Time domain indices from the spirogram were also calculated (14) and these are derived by considering the spirogram as made up of many small equal increments of expired volume each with their own time taken to be delivered, and this time is their transit time. The first few milliliters of volume delivered have a very short transit time, and the last few milliliters delivered have a much longer transit time. We derived the truncated first moment (₁), which is the mean of all the transit times of the spirogram standardized by the expired volume, and the second moment (₂), which is the mean of the square of all the transit times standardized by expired volume. The moment ratio (MR) is then given by

(2)

The MR up to 90% of FVC was chosen as it has previously been found to be a good index to reflect the shape of the flow-volume curve (14). A large MR occurs in a spirogram where the early part is steep, i.e., a high early flow, and the latter part of the spirogram is relatively flat and prolonged, i.e., with lower flows late in the maneuver. In UAO the expectation would be that the MR would be smaller than usual. In addition for all the patients with a goiter, that is, Sets 1 and 2, the ratios of forced expiratory and inspiratory flows at 50% of FVC (FEF₅₀/FIF₅₀) were calculated. These data were not available for the patients with COPD (Set 3).

Derivation of the Neural Networks

We used the neural network program NeuroShell 2 for Windows (Ward Systems Group, Frederick, MD) to develop networks to apply to our data. A series of neural networks with four layers of neurons were derived. Each network consisted of a layer of input nodes whose number matched the number of input variables, followed by two hidden layers with a back-propagation architecture (15, 16), and a layer of output nodes. The number of nodes in the hidden layers was adjusted to produce the best classification, which for the first hidden layer was always approximately twice the number of input nodes and for the second hidden layer was half the number of input nodes. In these networks the input layer received the input data and distributed it to the hidden layers. The input layer received no other inputs. Each node in the input layer connected to each of the nodes in the two hidden layers. Each node in the two hidden layers was connected to each node in the other hidden layer and also to each of the output nodes. Each connection between nodes allows the network to apply a weighting factor to the variable before it is received by another node. Each node sums the weights of its inputs and the sum determines its output to other nodes. Thus, the combined weighting of inputs may inhibit or enhance a node's output to other nodes. When a network is initiated the weights on each connection are randomly assigned values within certain limits, and as the network trains these weights are iteratively adjusted to give the closest agreement between the output of the network and the given answers for the training data set. The final weighting functions from the network can be used to identify those input variables that contributed most to the final categorization.

Each network was trained on a training data set and the resulting network was then tested on a separate test set of data. The success of the network in classifying the input data was expressed by calculating Cohen's kappa statistic (13). The training set of data was determined by the program, which randomly selected data from about one half to two thirds of the MEFV curves from each group of subjects. The remaining records were combined to produce the test set. The robustness of the networks was challenged by repeated analysis using different randomly allocated training sets, and by trying different network architectures.

Neural networks were then developed to classify patients into two groups, those with UAO (Set 1) and those without UAO (Sets 2 and 3). For the first network (NN1) the input indices chosen were the PEF, FEV₁, FVC, FEV₁%, the standardized residuals of these indices, the instantaneous flows, the flatness scores, and the moment ratio of the spirogram. A second network (NN2) was then developed, including FEF₅₀/FIF₅₀ as an additional input variable. Following the results from the above networks two further networks were developed; one (NN3) was derived using only the five inputs with the highest relative contributing factors from NN1 together with the FEV₁/PEF ratio as an input variable, and a further network (NN4) was derived exactly the same as for NN3 but without the FEV₁/PEF ratio, to see if this index made any contribution. The relative contribution factor for each of the input indices was calculated by summing the weights attached to all neurones leading from a particular input across all layers of the network. Finally, logistic linear regression was used to develop two models to classify patients into those with UAO (Set 1) and those without UAO (Sets 2 and 3). One model used the inputs developed above for NN3 and the other used the inputs for NN4.

Statistical analysis was performed using the SPSS version 6.0 for Windows; a level of 5% was taken as significant.

	RESULTS

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

Details of the age, FEV₁, and PEF for the subjects in the various data sets are shown in Table 1. As might be expected the patients with COPD tended to be older and had a lower FEV₁ and PEF than did either the patients with goiter but no UAO or those with goiter and UAO. When these data sets were split into the training and test sets for the neural networks the ages, FEV₁, and PEF were comparable between the sets.

The degree of certainty with which a decision was made about the configuration of the flow-volume loop by the two observers as the number and percentage of the 146 flow loops in each category is shown in Table 2. The mean (median) certainty of classification of all flow-volume loops for Observer 1 was 3.18 (3) and 2.77 (3) for the first and second readings, respectively, and was 2.95 (3) and 3.10 (3), respectively, for Observer 2. Observer 1 was significantly less certain about the decisions on the second classification (Chi square = 22.61, p < 0.001), whereas Observer 2 had no significant change in certainty (Chi square = 21.67, p = 0.17). The median score for each observer was the same, being "quite certain" about the decision.

The kappa scores for agreement with the correct assessment for all flow-volume loops were 0.50 for Observer 1 at both readings and 0.46 for Observer 2 at both readings. The kappa score for the repeatability of each classification by the two observers between scoring sessions was 0.86 and 0.63 for Observers 1 and 2, respectively. Agreement between the classification by the observers for each of the readings gave kappa scores between 0.58 and 0.68. The mean sensitivity and specificity for Observer 1 for the two readings were 75.5 and 73.7, respectively, and for Observer 2 were 83.7 and 71.1. Observer 1 obtained 31 and 20 false positives on the two readings, respectively, and 8 and 16 false negatives. Observer 2 obtained 28 false positives and 8 false negatives on each occasion.

The FEV₁/PEF ratio for patients with COPD, and goiter patients both with and without UAO, is shown in Figure 4. It can be seen that this index had considerable overlap between the groups. The results of classifying patients in the test set for the neural networks, logistic regression models, and by FEV₁/ PEF alone are shown in Table 3. Taking an FEV₁/PEF ratio of greater than 8 as indicating UAO (2) was a relatively poor classifier of patients into those with and those without UAO. The neural network using all the indices except FEV₁/PEF (NN1) produced a substantial improvement over FEV₁/PEF alone. The five indices with the highest relative contributing factors were in descending order of importance VCA3, VCP5P, VCA4, VCP1P, and MR_90%. Using these indices alone produced a network (NN4) that had a slightly lower accuracy than when using all indices (NN1). However, after adding FEV₁/PEF ratio to these five indices the resulting network (NN3) had a significantly greater ability to classify patients. These findings are presented in the style of a receiver operator characteristic (ROC) plot in Figure 5. Redefining the network by adding FEV₅₀/ FIF₅₀ to all the indices (NN2) but without the data for patients with COPD did not improve the ability of the original neural network to classify the data, with FEF₅₀/FIF₅₀ being ranked 25th among the relative contributing factors for NN2.

View larger version (17K): [in this window] [in a new window]   Figure 4.   A plot of the FEV1/PEF ratio for subjects with goiter and UAO, those with goiter but no UAO, and those with COPD. The mean values are shown as broad bars. In each section some subjects had identical values.

View larger version (15K): [in this window] [in a new window]   Figure 5.   A receiver operator characteristic (ROC) style of plot for the various methods tried to identify the subjects with upper airway obstruction. The higher and farther to the left a method is placed in the plot, the better it is at correct identification of UAO.

Building a linear logistic regression model using only the five highest contributing indices produced a model with no significant interactions at the 5% level of significance. The VCP1P and VCP5P were the only indices with significant coefficients, and the resulting model was less sensitive than the neural networks in identifying UAO. Redefining the model by adding FEV₁/PEF did not alter the performance of the logistic regression model, but the indices with significant coefficients were then VCP1P and FEV₁/PEF and their interaction. In both the neural network and logistic regression based models the false positive cases for UAO were all patients who had a goiter but no UAO, that is, the neural networks never falsely identified COPD as UAO.

	DISCUSSION

TOP ABSTRACT INTRODUCTION METHODS RESULTS DISCUSSION REFERENCES

We have shown that analysis of flow-volume curves with a neural network was a more reliable and accurate predictor of upper airway obstruction caused by goiter than were trained human observers. The shape of the flow-volume curve associated with upper airway obstruction has been previously described (2, 3, 5, 6, 17) and clinicians with an interest in pulmonary physiology learn how to recognize these patterns. Although in some cases this pattern recognition is quite easy, in a group of patients with lesser degrees of UAO the pattern may not be so easily recognized. Our observers were reasonably consistent in their classification, but they were not as good as the neural network in agreeing with the accepted classification, the latter being based not only on the flow loop but also on the available clinical and physiologic data.

The clinical importance of our findings relies on our original classification being correct. In an earlier study (2) we had shown that with a mixed range of symptomatic patients our human expert's identification of patients with and without upper airway obstruction was verified by postoperative flow-volume loop testing, with what appeared to be two false positives, no false negatives, 14 correct negatives, and 27 correct positive identifications being made. We therefore believe that in the current study the data and their original classification, which were presented to the neural network, were a reasonably accurate set of data. We have previously found that the addition of ultrasonography and orthogonal plain radiographs did not help predict the presence of UAO (2). In trying to define a better original classification it might be helpful to have measurements of airway resistance and CT scans of the upper airway. The former may not distinguish the level of any increase in resistance, and the latter was thought to be unwarranted on grounds of cost-benefit, logistics, and irradiation. Even if there were some false calls in the data set that the network was trained on and tested on, we have shown that the neural network still performed better at identifying the various sets than an expert with knowledge of all the clinical data. If a perfectly validated data set was available for the purposes of training the neural network, it is likely that the performance of the network might be better still.

A neural network is an instrument that allows patterns of difference between defined groups to be learned and then applied to new test data to see if the learned pattern offers a better discrimination than that of the human observer (15). Neural networks are not governed by any laws of causality or association between data. They excel at pattern recognition by coping with noisy data, and during training they assess all interactions between input indices. They are thus able to classify data when the discrimination boundary between categories is highly complex. If a network trains for too long on a set of data then the result may not be applicable to, or helpful in analyzing a different set of data. To avoid this problem the program we used limited the training to the point at which the error rate for detection of UAO in both the training set and its application to the test set were found to be at a minimum. By prolonged training on the training set it is possible to continue to get improvements in error rate, but a point is often reached when the applicability of this network to other data is limited because of overfamiliarity with the training set. At this point the error rate in detecting UAO in the remaining test set starts to rise with further training. By routinely testing for this effect the program can terminate the training to achieve an optimal network that can be used on other data. When our networks were challenged by running on different training sets we found them to be robust in giving the same indices as best predictors. Also, the network architecture was robust and gave the optimal convergence to a satisfactory solution.

The application of the network to data other than that on which we have used may potentially be misleading. We have tested the ability to distinguish UAO from airflow limitation caused by COPD and from normal subjects. We do not have a sufficiently large data set of flow loops from patients with restrictive disease to allow these to be used in our analysis, and so it is possible it would not perform as well if these subjects were included. Using the network on loops that have been obtained in a suboptimal way, either because of poor laboratory quality control or poor recording characteristics of the device used, would also degrade the utility of the network.

In this study it was a neural network using a limited range of flatness scores, the moment ratio, and the FEV₁/PEF ratio as inputs that was the most accurate classifier of UAO. The moment ratio to 90% of FVC has previously been proposed as an index from the spirogram that is sensitive to the shape of the spirogram and flow-volume curve (14). It has been found to be highly repeatable within subjects and to be sensitive to changes in the shape of the spirogram (18) when looking for the effects of cigarette smoking on airflow from the lungs. In UAO, where there is a period of relatively constant flow early in the forced expiratory maneuver, the moment ratio will reflect the consequent change in shape of the early part of the spirogram and flow-volume curve to be lower than normal and thus has been found to be helpful in detecting UAO. The flatness scores chosen by the network were the length of chords taken at 1.5 and 2.0 L/s below PEF, and at 95 and 75% of the recorded PEF. The chords at decrements of predicted PEF were not as informative as these. The FVC is not usually affected in patients with UAO, and so it is not clear why for two of the above chords the neural network preferred them standardized to recorded FVC and for the others to be standardized to predicted FVC. No single chord length was sufficiently effective in our data to be able to use it on its own to help predict the presence of UAO; however, a linear combination of these flatness indices did improve classification of the data compared with the single index of FEV₁/PEF alone.

Our earlier study (2) used the comparison with the change in lung function after surgical intervention to verify whether patients had surgically amenable UAO. By this criterion the inspection of the flow-volume loops had good sensitivity (100%) and fair specificity (78%), with an overall accuracy of 91%. In this study we were unable to make a postoperative comparison for verification, but the same observer as in the earlier study used all the available clinical and physiologic data to make the initial classification of the flow-volume loops in this study. The repeated classification of the curves by the two observers produced kappa scores suggesting a fair degree of agreement and repeatability between observers, but without access to all the clinical data, the sensitivity and specificity of the observers' assessment of the flow-volume loops was not as good as the neural network. The variability of this assessment will partly be due to the nature of the flow-volume loops in this study, with approximately one-third being found by the observers to be at least moderately difficult to classify.

The traditional index of FEV₁/PEF ratio as a single discriminator performed relatively poorly compared with classification using multiple indices based on neural networks or logistic regression. The FEV₁/PEF ratio did not separate patients who had goiter and UAO from those who did not, although this ratio could more easily separate the patients with UAO from those with COPD. Using the originally suggested limit of FEV₁/PEF greater than 10 to define UAO (10), rather than using a limit of 8 (2), would improve specificity at the cost of a decreased sensitivity, a finding in agreement with our earlier study (2).

We have shown that for the detection of UAO caused by goiter a neural network using a limited combination of indices from a maximal forced expiratory maneuver is better at correctly classifying the data than the interpretation of the flow-volume curves by human observers. Four of these indices were novel scores of flatness derived from the flow-volume loop, and the utility of these indices and the method using a neural network should now be tested prospectively in a clinical setting. The results from this network could be put directly into computerized pulmonary function equipment to help identify the presence of UAO. Supplied with the relevant indices the network can be used to make a classification as to whether UAO is thought to be present in a particular subject. This classification would clearly not be infallible, but it might aid clinicians in their choice of further investigations and management strategy. On the basis of the subsequent clinical outcome the network can than be further trained using increasingly more robust and refined training sets. This approach may improve the ability to diagnose UAO in the absence of local expert opinion, using the relatively simple test of a flow-volume loop, which can be undertaken on all patients with a goiter.