Classification of lung nodules based on the DCA-Xception network.

BACKGROUND: Developing deep learning networks to classify between benign and malignant lung nodules usually requires many samples. Due to the precious nature of medical samples, it is difficult to obtain many samples. OBJECTIVE: To investigate and test a DCA-Xception network combined with a new data enhancement method to improve performance of lung nodule classification. METHODS: First, the Wasserstein Generative Adversarial Network (WGAN) with conditions and five data enhancement methods such as flipping, rotating, and adding Gaussian noise are used to extend the samples to solve the problems of unbalanced sample classification and the insufficient samples. Then, a DCA-Xception network is designed to classify lung nodules. Using this network, information around the target is obtained by introducing an adaptive dual-channel feature extraction module, and the network learns features more accurately by introducing a convolutional attention module. The network is trained and validated using 274 lung nodules (154 benign and 120 malignant) and tested using 52 lung nodules (23 benign and 29 malignant). RESULTS: The experiments show that the network has an accuracy of 83.46% and an AUC of 0.929. The features extracted using this network achieve an accuracy of 85.24% on the K-nearest neighbor and random forest classifiers. CONCLUSION: This study demonstrates that the DCA-Xception network yields higher performance in classification of lung nodules than the performance using the classical classification networks as well as pre-trained networks.

Keywords: Lung nodule classification; Wasserstein Generative Adversarial Networks (WGAN); Xception; convolutional attention module; classifier

Lung cancer is more common and has a higher mortality rate than most cancers, posing a significant threat to people's health and lives [[1]]. According to statistics, the 5-year survival rate of lung cancer patients is only 19% [[2]]. Because by the time their bodies exhibit clinical signs, they are already in the middle or late stages of the disease, and cancer has spread to organs other than the lungs, resulting in higher treatment costs and difficulty. Lung cancer mortality can now be reduced by 20% with early CT screening [[3]]. However, the screening process necessitates a high level of focus and expertise. For less experienced radiologists, this leads to highly variable detection rates and an increase in the rate of false-positive detection [[5]]. Therefore, the use of computer-aided diagnosis systems can avoid these problems.

For the problem of classifying benign and malignant lung nodules, some researchers have chosen to extract features manually, which are usually size [[6], [8]], texture [[9], [11]], histogram of oriented gradients (HOG) [[13], [15]], etc., and then classify them by classifiers such as support vector machines (SVM) [[16]], random forest [[17]], K-nearest neighbor (KNN) [[18]]. Because of the advantages of convolutional neural networks (CNN) in automatically extracting typical features, more and more researchers choose CNN to process medical images [[19]]. Liu et al. [[21]] used CNN networks to extract features from pulmonary nodules, dimensionality reduction of the extracted features using principal component analysis, and finally classified pulmonary nodules using SVM. Chae et al. [[22]] proposed a deep learning-based malignancy prediction model (CT-lungNet), and initially evaluated its performance and usefulness to human evaluators. Naik et al. [[23]] used FractalNet to classify lung nodules and demonstrated excellent classification performance, validated on the LUNA dataset, achieving 94.7%, 90.41%, 96.68%, and 0.98 scores for accuracy, specificity, sensitivity, and area under the receiver operating characteristic curve.

As the study progressed, the researchers found that classification lung nodules using deep neural networks required many samples. Due to the precious nature of medical images, which are difficult to obtain in large quantities, researchers have come up with the following solutions. Zhao et al. [[24]] proposed an Agile CNN model based on LeNet and AlexNet to overcome the challenges of small medical image databases and small nodules. Zhao et al. [[25]] implemented three strategies, including modifying some state-of-the-art CNN architectures, integrating different CNNs, and using migration learning. Nobrega et al. [[26]] explored the performance of deep transfer learning in the classification of lung nodules malignancy. It was found that the pairing of ResNet50 as a feature extractor and SVM as a classifier worked best. In addition to the above methods, Xie et al. [[27]] proposed a semi-supervised adversarial classification (SSAC) model which can be trained by using labeled and unlabeled data. Through collaborative learning based on multi-view knowledge, the MK-SSAC model was evaluated on the LIDC-IDRI dataset with an accuracy of 92.53% and an AUC of 0.958. Onishi et al. [[28]] used many nodule images generated by GAN to train the DCNN and then fine-tuned it using a small number of actual nodule images to enable the DCNN to distinguish between benign and malignant nodules. This method determined 66.7% of benign nodules and 93.9% of malignant nodules. Table 1 shows the existing methods for classifying lung nodules and their classification performance.

Table 1 Overview of the existing classification methods

In this paper, to address the problems of insufficient samples and class imbalance, first CT slices are randomly cropped. Then, a Wasserstein Generative Adversarial Network (WGAN) with conditions is used to balance the class of the samples, and finally, five data augmentation methods such as flipping, rotating, and adding Gaussian noise are used to extend the dataset. As can be seen from Table 1, for the lung nodule classification problem, many researchers have used pre-trained networks to classify lung nodules. Still, the scalability, flexibility, and generalization of pre-trained networks are poor compared to customized networks. Therefore, in this paper, a DCA-Xception network is designed, which has higher flexibility and better classification performance than the fine-tuned pre-trained network. The network obtains information around the target through an adaptive dual-channel feature extraction module. The convolutional block attention module enables the network to learn effective features in a targeted manner.

Figure 1 shows the workflow of the DCA-Xception network for lung nodule classification. As can be seen in the figure, the regions of interest are first extracted from the collected data, followed by expanding the data using WGAN with conditions and data augmentation, followed by inputting the patches into the network for classification of lung nodules. There are two ways of classification, the first one directly uses the improved network to classify lung nodules, i.e., the fully connected layer acts as a classifier. In a second way, the output of the last global average pooling layer of the network is used as the feature result. Subsequently, the feature input for the six classifiers (logistic, multinomial Bayesian [[35]], multilayer perceptron (MLP) [[36]], SVM, KNN, and random forest) for training and testing, and finally, the classifier that works best with the network proposed in this paper is selected for classification. The first way is more convenient and consumes fewer computational resources, while the second way is complex but is more accurate in classifying lung nodules. In the application, the appropriate classification way can be chosen according to the needs and conditions.

Graph: Fig. 1 Workflow based on DCA-Xception for lung nodule classification.

The dataset used in this paper is the LIDC-IDRI public dataset, in which four physicians classify the malignancy of lung nodules. The degree of malignancy is classified into five categories, where categories one and two indicate benign, category three indicates indeterminate benignity, and categories four and five indicate malignancy. In this paper, nodules that are jointly identified by three or more physicians are selected, and their malignancy is determined by the average of the physician-labeled categories. A nodule is considered benign when the mean value of that nodule category is less than or equal to 2.5, nodules between 2.5 and 3.5 are removed, and nodules greater than or equal to 3.5 are considered malignant. A total of 326 nodules are selected, of which 274 nodules (154 benign and 120 malignant) are used for the training and validation sets, and 52 nodules (23 benign and 29 malignant) are used for the test set.

To make full use of the data, all CT scans slices covering the nodules are utilized, and each slice is treated as a sample. The total number of the generated slices and their distribution are shown in Table 2 Initial. In addition, patches with a size of 64×64, including the pulmonary nodules, are extracted after establishing the location of the pulmonary nodules based on the XML file as well as ground truth labels. The nodules are not concentrated in the center of the patches but are located at arbitrary positions. Four patches are extracted for each slice of 274 lung nodules using the above method to obtain lung nodules in different contexts. The number of patches obtained is divided into the training and validation sets by 9:1, as shown in Table 2 Original. Patches are extracted once for malignant nodule slices in the test set, and benign nodule slices are extracted in multiple rounds until their number approaches the number of malignant nodule patches, as shown in Test in the last column of Table 2.

Table 2 Specific settings for each stage of the data set

In CT images, the number of slices of malignant nodules is usually much larger than the number of slices of benign nodules, which results in a smaller sample size than that of malignant nodules even though the number of benign nodules is high. To balance the sample, more benign nodules or fewer malignant nodules are needed. However, because access to medical images is protected by personal privacy and laws, it is difficult to obtain many samples. If the number of malignant nodules is reduced, the diversity of samples will weaken. Using GAN [[37]] can solve the above problems, but the model tends to collapse during training, and the generated sample classes are often uncontrollable. Therefore, in this paper, WGAN [[38]] is chosen to generate patches to make the network more stable during the training process, and constraints are added to enable the network to generate patches of specified classes. The structure of WGAN with conditions is shown in Fig. 2. The number of patches after class balancing is shown in Balance in Table 2, where the number in parentheses is the number of patches generated by the WGAN with condition.. The generated patches are shown in Fig. 3(b).

Graph: Fig. 2 Structure of WGAN with conditions.

Graph: Fig. 3 Example of patch images.

To support deeper neural network training, as well as to avoid overfitting, this paper uses five data augmentation methods to expand the dataset again. The data augmentation methods include flipping up and down, flipping left and right, rotating counterclockwise (90°, 180°, 270°), randomly changing brightness, and adding Gaussian noise. The number of patches after data augmentation is shown in Table 2 Augmentation. An example of using five data augmentations is shown in Fig. 3(c).

Compared to classification networks of the same depth, the Xception network utilizes depth-wise separable convolution, which allows the network to have a smaller number of parameters at a better classification performance. However, the network is prone to misclassify the class of lung nodules due to its lack of ability to combine information around the target and extract effective features. Therefore, in this paper, the Xception network is improved to enhance its classification performance for lung nodules. The improved network structure is shown in Fig. 4. The main improvements are the addition of an adaptive dual-channel feature extraction module and a convolutional block attention module to the middle flow of the Xception network.

Graph: Fig. 4 DCA-Xception structure.

Only the 3×3 depth-wise separable convolution is utilized in the middle flow of the Xception network, which obtains single feature information and cannot effectively combine the information around the target. Therefore, in this paper, an adaptive dual-channel feature extraction module is added to the network, whose structure is shown in middle flow part A in Fig. 4. As seen from the figure, the module first performs N×N and 3×3 dual-channel feature extraction on the input features. It then adds adaptive coefficients α to the features extracted by the convolution kernel of N×N. Then the features extracted by the dual-channel are combined, the information is integrated, and the network parameters are reduced by 1×1 convolution. Finally, new features are extracted with a 3×3 depth-wise separable convolution. This paper uses two sets of adaptive dual-channel modules with N of 1 and 5, respectively. The network can use different receptive fields to obtain features and efficiently combine information around the target. Also, the residual structure is used to prevent gradient disappearance and explosion.

The network learns intricate features during the training process, but not all these features are valid. Therefore, the convolutional block attention module [[39]] is introduced, which can increase the weight of useful features and weaken the weight of useless features during the training process and has the effect of learning for useful features. The structure of the convolutional block attention module is shown in Fig. 4 CBAM. As can be seen in part B of the figure, first, 1×1 convolution is used to reduce the parameters, and then two layers of 3×3 depth-wise separable convolution are used for feature extraction. Finally, the spatial attention module and the channel attention module are used to divide the weights of the learned features.

In addition to the above improvements, to inherit the advantages of the small parameters of the Xception network, some adjustments are made to the parameters of the improved middle flow, mainly the number of repetitions of the middle flow is adjusted from 8 to 4, and the number of channels is reduced from 728 to 512.

The patch size of the input model is 64*64, and the data is normalized before input to improve the network convergence speed. The algorithm is experimented using the Adam optimization algorithm with a batch size of 64 and a learning rate of 0.001 on a GPU (NVIDIA Tesla K80 16GB). Adaptive decay of the learning rate is chosen. The loss values of the validation set are detected, and the learning rate decayed tenfold when two consecutive epochs (794 iterations per epoch) are no longer decreasing. To save computational resources and to prevent the network from overfitting, the early stopping strategy is chosen. The network stops training when the loss value of the validation set no longer decreases for five consecutive epochs.

To be able to backpropagate the gradients corresponding to each category in a stable manner, the problem of gradient disappearance during back propagation of the network is effectively solved. Therefore, in this paper, cross-entropy is used as the loss function, as shown in Equation (1). The smaller the loss function is, the better the classification performance of the model and the stronger the robustness.

(1) $\mathrm{Loss}=\frac{1}{N}\sum _{i=1}^N(\mathit{gi}·ln(\mathit{pi})+(1-\mathit{gi})·ln(1-\mathit{pi}\left)\right)$

In the formula, gi is the true class of sample i, and pi is the prediction of the network for sample i.

The evaluation metrics accuracy (ACC) Equation (2), F-Score Equation (3), specificity (TNR) Equation (4), sensitivity (TPR) Equation (5), and area under the receiver operating characteristic curve (AUC) Equation (6) are used to measure the classification performance of the network, and the formulas are shown below.

(2) $\mathit{Accuracy}=\frac{(\mathit{TP}+\mathit{TN})}{(\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN})}$

(3) $F-\mathit{Score}=2.\frac{\mathit{TP}}{\mathit{FP}+2\mathit{TP}+\mathit{FN}}$

(4) $\mathit{TNR}=\frac{\mathit{TN}}{(\mathit{TN}+\mathit{FP})}$

(5) $\mathit{TPR}=\frac{\mathit{TP}}{(\mathit{TP}+\mathit{FN})}$

where TP, TN, FP, FN indicate the number of true positives, true negatives, false positives, false negatives, respectively.

(6) $\mathit{AUC}={\int }_0^1{t}_{\mathit{pr}}\left(f_{\mathit{pr}}\right){\mathit{df}}_{\mathit{pr}}=P(X_p>X_n)$

The true positive rate tpr is a function of the false positive rate fpr along the receiver operating characteristic curve, and Xp and Xn are the confidence scores for a positive and negative instance, respectively.

Figure 5 shows the changes in the loss values and accuracy rates of the training and validation sets of the model proposed in this paper during the training process. From the figure, it shows that the accuracy rates of both the training and validation sets are increasing smoothly. The loss values of the training set decrease smoothly and converge gradually, but the loss values of the validation set show slight fluctuations and level off gradually after the 7th epoch. The lower the loss value is, the better the generalization performance of the model, so the weights under the 5th epoch are chosen to test the model's performance.

Graph: Fig. 5 Curves of loss and ACC.

Table 3 shows the experimental results of the ablation experiments. As can be seen from Table 3, A has poor testing results on the original dataset using the original model, which has less than 50% TNR. B uses WGAN with conditions for category balancing on the dataset while expanding the sample size by a small margin. Since the stationary features generated by WGAN make the network easier to learn, therefore, all the metrics in the experimental results are improved except TPR, in which the TNR is improved by 23.17%. C used five data augmentation methods, which greatly increase the amount of data. The network is less likely to be over-fitted and can learn more features. Compared with B, the AUC is improved by 0.098. D builds on the model of C by cutting the middle flow to 4 times and introducing an adaptive dual-channel feature extraction module.

Table 3 Ablation experiments

Thus, the design of the adaptive dual-channel feature extraction module enables the model to combine the surrounding information when classifying the target and classify the lung nodules more accurately. In this case, the model achieves the optimal value of each index compared with A, B, and C. E is based on D and introduces the convolutional block attention module, enabling the model to learn features more accurately through spatial attention and channel attention module. This improvement makes the classification performance of the model improve again. Among them, the TPR improves by 4.57% to 92.89%.

In this paper, the without pre-trained LeNet [[40]], AlexNet [[41]], VGG16 [[42]], InceptionV3 [[43]], ResNet50 [[44]], and Xception [[45]] networks are selected as the comparison networks for the classification way 1 proposed in this paper. To make the models converge faster, all models include batch normalization (BN) layers. The above models' training strategies are the same as the network training strategies in this paper. The experimental results are shown in Table 4, where the bolded font indicates the optimal values. Figure 6 shows the receiver operating characteristic (ROC) curve of each network. It can be seen in combination with Table 4 and Fig. 6 that the proposed method achieves the optimal value in all the indexes except TNR, which fully demonstrates the effectiveness of this network for lung nodule classification. In addition, the resulting confusion matrix of DCA-Xception in this paper is shown in Fig. 7.

Table 4 Comparison with typical classification networks

Graph: Fig. 6 Comparison of the ROC curves of classification way 1 and classical classification network.

Graph: Fig. 7 Confusion matrix.

In this paper, the pre-trained networks VGG19, RestNet101, InceptionV3, MobileNetV2 [[46]], Xception, Inception-ResNetV2 [[47]], and DenseNet121 [[48]] on ImageNet are selected as feature extractors. Then logistic, multinomial Bayesian, multilayer perceptron (MLP), SVM, KNN, and random forest classifiers are used to perform the classification. The training steps are as follows: First, a fully connected layer is added after the convolutional base of the pre-trained model and fine-tuned. Then the fully connected layer is removed, and the output of the global average pooling layer is used as the result of the features. These features are then input to each classifier for training. Finally, a test set is used to verify the classification performance of the model under each classifier and to compare it with the proposed classification way 2 in this paper.

The experimental results are shown in Table 5, where the best values of each metric in all models in bolded font. Figure 8 shows the ROC curve for each pre-trained network. Combining Table 5 and Fig. 8, it shows that the fine-tuned pre-trained network has better TPR, but its TNR is lower. The proposed method achieves the best values for ACC, AUC, and F-Score, indicating that the model outperforms the above pre-trained models in classification and achieves a better balance of TPR and TNR.

Table 5 Comparison with pre-training network

Graph: Fig. 8 Comparison of the ROC curves of classification way 2 and pre-trained network.

Table 6 shows the changes in the number of parameters, training speed, and inference speed after adding the adaptive dual-channel feature extraction module and the convolutional block attention module. As can be observed in the table, the improved model improves accuracy and AUC, with ACC increasing by 4.58% and AUC increasing by 0.035. Due to the reduced number of middle flow repetitions, the number of parameters of the model and the model complexity are slightly reduced. However, the training speed and inference speed of the model become slower due to more element-wise operations. For the classification of lung nodules, the importance of classification accuracy is greater than inference speed. Therefore, this model is acceptable to lose a small amount of inference speed while improving the classification accuracy. In addition, this paper explores the effect of the variation of N in the dual-channel feature extraction module on the classification performance. From Table 6, it can be seen that when N is both 1 or 5, the network obtains limited information around the target, which improves the classification performance of the network less. When N is 1,7 or 1,5, the network can perform better in classifying lung nodules because it receives more information about the target. Combining the indicators, the best classification performance of the network for pulmonary nodules is obtained when N is 1,5.

Table 6 Number of participants and speed

Through a series of experiments described above, it is shown that the proposed method in this paper produces satisfactory results on the LIDC-IDRI dataset. Compared with classical classification networks and pre-trained networks in the same dataset, it has better performance in classifying benign and malignant lung nodules while maintaining a smaller number of parameters. In addition, the model proposed in this paper has better structural flexibility. The number and size of the convolutional kernels of the adaptive dual-channel feature extraction module can be changed to cope with different data sets. In this paper, two classification ways are utilized. The main difference is that classification way 1 utilizes fully connected layers for classification, while classification way 2 utilizes various classifiers for classification. Experiments show that using classifiers for classification will improve the accuracy of classification. Therefore, the classifiers can be optimized to further improve the classification accuracy. One of the limitations of this paper is the exclusion of benign and malignant easily confused pulmonary nodules between 2.5 and 3.5, so the classification performance of the proposed method for this type of nodules needs to be further investigated. In addition, since pulmonary nodule lesions are present in multiple CT slices, the method in this paper only targets the lesion regions in individual slices, ignoring the association between images of adjacent slices. Therefore, using the pixel association between lung nodule lesions in adjacent slices to improve the classification accuracy will be the next research focus.

In this paper, to address the problem of insufficient samples and unbalanced categories of lung nodules, five data augmentation methods and WGAN with conditions are utilized to expand the samples. An adaptive dual-channel feature extraction module and a convolutional block attention module are introduced in the middle flow of Xceotion to improve the classification performance of the model. The computational complexity, number of parameters, training speed and, inference speed of the DCA-Xception network are researched. The lung nodule classification performance is compared with that of classical classification networks and pre-trained networks to verify the effectiveness of the improved network. The experimental results show that the network outperforms the traditional classification network and the pre-trained network in classifying lung nodules. In summary, the method proposed in this paper, which has a better performance on the lung nodule classification task, can provide effective diagnostic decision support for physicians.

This work was supported by the National Natural Science Foundation of China (NO.51975170), Youth Innovation Fund of Heilongjiang Academy of Sciences (NO.CXJQ2020WL01), Basic Applied Technology of Heilongjiang Institutes Research Special Project (NO.ZNJZ2020WL01), Natural Science Foundation of Heilongjiang Province (NO.LH2019F024).