Pattern recognition of DC partial discharge (PD) receives plenty of attention and recent researches mainly focus on the static characteristics of PD signals. In order to improve the recognition accuracy of DC cable and extract information from PD waveforms, a modified deep belief network (DBN) supervised fine-tuned by the adaptive moment estimation (ADAM) algorithm is proposed to recognize the four typical insulation defects of DC cable according to the PD pulse waveforms. Moreover, the effect of the training sample set size on recognition accuracy is analyzed. Compared with naive Bayes (NB), K-nearest neighbor (KNN), support vector machine (SVM), and back propagation neural networks (BPNN), the ADAM-DBN method has higher accuracy on four different defect types due to the excellent ability in terms of the feature extraction of PD pulse waveforms. Moreover, the increase of training sample set size would lead to the increase of recognition accuracy within a certain range.
Keywords: DC cross linked polyethylene (XLPE) cable; partial discharge (PD); restricted Boltzmann machines (RBM); deep belief network (DBN); adaptive moment estimation (ADAM)
HVDC (High Voltage Direct Current) transmission has the advantages of low line cost, not being restricted by synchronous stability problems, thus the project of DC cable has developed rapidly in recent years [[
The accumulation of space charge inside the polymer leads to local electric field distortion, which may result in partial discharge (PD), especially in insulation defects. Thus, the PD detection is often used for XLPE cable detection to evaluate the operating condition of the cable. There are lots of literatures about PD researches of AC XLPE cable, but few standards or recognized diagnose methods for DC cable [[
In this paper, a modified DBN algorithm based on DC PD pulse waveforms is proposed to achieve the pattern recognition of DC cable insulation defects. First, the PD signals of typical insulation defect cables are collected through the DC experimental platform. Second, the PD pulse waveform is pre-processed using the Canny algorithm and used as an input sample for the classification model. Third, a DBN recognition model optimized by the adaptive moment estimation (ADAM) algorithm is built to achieve the pattern recognition of different defect types. Finally, compared with the artificial feature recognition methods, the effect of classification methods on various defects and the effect of training sample capacity on the classification models are analyzed. The experimental results show that the proposed method improves the accuracy of the recognition of DC XLPE cable insulation defects.
The deep belief network is a non-convolution generation model proposed by Hinton et al. in 2006 which solves the problem that the depth model was difficult to optimize. The deep belief network consists of a multi-layered restricted Boltzmann machine (RBM). It uses the contrastive divergence (CD) algorithm for unsupervised training of RBM and then uses supervised training for tuning of the entire DBN network.
The RBM is an undirected probability graph model consisted of the visual layer and the implied layer group, where there is no connection between the cells in the visual layer or the implied layer. The RBM is an energy-based model consisting of a visual layer v and a hidden layer h, with RBM's energy function [[
(
where θ = {w = (w
The joint probability distribution is specified by the energy function [[
(
where Z is a normalized factor called an allocation function:
(
The state of the hidden elements in RBM is independent of each other, and when given the visible unit v, the probability that the hidden unit h is activated (set to 1) is [[
(
Similarly, when the state of hidden unit h is determined, the probability that the visible unit v is activated is [[
(
where sigmoid = (1 + exp(−x))
For the RBM model given the number of explicit elements and hidden elements, it is necessary to determine the parameter θ through training, where the training goal is to make the data reconstructed from the RBM model under the control of the parameter θ as consistent as possible with the given training sample data. Because the fractionation function of RBM, Z
The CD algorithm first initializes the parameters θ randomly, uses the training sample as a visible unit v
(
where
In the actual training, due to the large sample size, the random sampling of training samples for RBM was trained in several small quantity batches to improve the computational efficiency. Suppose the number of the random sample size is N, the update formula for the k-th training is:
(
The classification model based on the DBN is stacked with several RBMs and consists of an input layer, several hidden layers and an output layer consisting of the Softmax classifier. The structure is shown in Figure 1.
Each of the two adjacent layers in the input layer and the hidden layer constitutes an RBM, as shown in Figure 1. The RBM
In DBN, RBM network is unsupervised trained layer by layer. First, using the CD algorithm training the input data v
RBM training is carried out independently, so the model parameters are optimal in RBM locally. Therefore, the parameters need to be fine-tuned at the end of the pre-training phase. For a given set of training samples {x, y}, the relationship between input and output can be expressed as:
(
where f is a nonlinear function; x
Use the average of the cross entropy of the predicted and actual values as the function of error loss:
(
where N is the number of the training samples and y
The supervised fine-tuning of DBN usually uses the gradient drop method and conjugate gradient descent method. But it is difficult to select the appropriate learning rate and seems to converge to local optimal. The adaptive moment estimation (ADAM) algorithm was proposed by Kingma in 2015 and has excellent performance in dealing with sparse gradients and unsteady states, requiring only a small amount of tuning [[
(
Then, calculate the first-order moment deviation
(
The network parameters are updated:
(
where α is the step length, τ is the stable constant.
ADAM algorithm parameters are set according to the reference [[
Based on the common insulation faults in the actual operation of XLPE cables, this paper designs four typical defect models: (
The experimental system is shown in Figure 3. The experiment was carried out in the shielding chamber and the environment remained in the range of (293 ± 5) K. The high-voltage DC source used in the experiment is a 5 kVA/200 skV DC generator, C1 and C2 are 200 kV/100 pF DC divider, C3 is 200 kV/10,000 pF filter capacitor, R1 is 10 kΩ protective water resistance, C4 is 150 kV/300 pF coupling capacitor. This experiment used the HFCT detection system to detect partial discharges. The passband of the Rogowski coil is 3 MHz to 50 MHz. The Lecroy high-speed oscilloscope has a bandwidth of 2 GHz and a maximum sampling rate of 20 GS/s. The HFCT detection coil is clamped on the ground wire of the copper shielding layer of the cable, and then connected to the oscilloscope through the PD signal acquisition module. The rise time of PD pulse current is usually nanosecond order, the pulse duration is about 1000 ns. The sampling rate of the oscilloscope is set to 200 MSa/s, which meets the requirements of the Nyquist sampling theorem. Each cable defect model is tested under constant DC voltage, maintain the applied voltage of the cable at partial discharge inception voltage. In the pressure test, a constant voltage test was carried out at five voltage levels. The voltage level gradient was about 4.5 kV, and each voltage gradient was maintained for at least 20 min.
The four defect models were connected into the system individually, and then slow boost the voltage when PD was relatively stable. The PD pulse waveform signal data was collected through HFCT. The measured pulse waveform signal is shown in Figure 4, the oscilloscope acquisition PD signal contains redundant information before and after the pulse time. Therefore, this paper introduces the modified Canny edge detection operator to pre-process the pulse waveform and intercepts the part of the red area in the figure so as to maximize the information efficiency.
Canny edge detection has the advantages of strong anti-noise interference, easy detection of weak edges [[
After determining the starting point of the discharge waveform, the discharge waveform is intercepted according to the length of the waveform and the sample rate, which is set to 600 in this article. Due to the different stable discharge voltages of different defects, the amplitude is normalized after intercepting the waveform.
The pre-processed PD waveform of four typical defects is shown in Figure 5. As shown in figure, the discharge current of C1 and C2 have 3 or 4 fluctuation approximately, and the amplitude decays faster, where the decline of current amplitude fluctuation of C2 decrease rapidly. The current waveforms of C3 and C4 are similar, have about 6–7 triangularly fluctuations, and the current amplitude decay is slow, where the current of C4 becomes an irregular violent jitter at a later stage.
The pre-processed current waveform is used as sample data, and the sample dimension is the signal length intercepted. A total of 6400 samples were collected, where 1600 samples of discharge signals for each type of insulation defect. In addition, 14 characteristics of the PD current waveforms are extracted, such as skewness degree S
The PD pattern recognition step of DC cable based on the ADAM-DBN are shown as follows:
- (
1 ) The original data are pre-processed and divided proportionally into training sets and sample sets. - (
2 ) Construct the DBN recognition model, pre-train it with CD algorithm, and get the pre-trained network parameters of the identification model. - (
3 ) Using the ADAM algorithm, the DBN model is supervised trained by training sample label, fine-tuning the network parameters to make the model optimized. - (
4 ) Use the test data set as input to inspect the trained DBN model.
Accuracy and recall are used as indicators to measure the effectiveness of various methods of diagnosing insulation defects. In multi-category problems, accuracy represents the proportion of all samples correctly predicted to the total sample:
(
Recall rate represents the proportion of the actual type of sample that was correctly determined for that type:
(
where, X
Through the analysis of accuracy and recall rate, we can not only evaluate the overall effectiveness of the pattern recognition method, but also examine the specific identification effect of the pattern recognition method for each defect, and analyze the application of the algorithm to different defects.
The initial bias of parameters in the DBN model are set to 0, and the initial weight uses the random number generated by the Gaussian distribution, where the expectation of Gaussian distribution is set to 0, the standard deviation is set to the inverse of the mean square root value of the number of cells in the visual layer, the batch size of the pre-training stage is 100, the RBM learning rate is 0.1, and the learning cycle of the fine-tuning stage is preset 200. Because the hidden layer structure and learning cycle have a great influence on the recognition accuracy, this paper determines the specific value through experiments.
The number of hidden layers of the model and the number of hidden units per hidden layer are obtained by the enumeration method. Thereby, first, a hidden layer is set for the model, the number of hidden units is gradually increased, the average accuracy of 10 experiments is recorded as a performance indicator, then the optimal value of the number of hidden units is determined. Moreover, add one more hidden layer and choose the optimal number of hidden units of second hidden layer, until the validation score is no longer improved.
The relationship between the number of hidden units and the validation score is shown in Figure 6, where N indicates that there is one hidden layer and the number of hidden units is set to N, and 50-N indicates that there is two hidden layers and the number of hidden units of each layer are 50 and N respectively. Considering the computational efficiency, the network structure of this paper contains two hidden layers, where the numbers of hidden units in the first and second layers are both set to 50.
After determining the structure of the hidden layer, the effect of the learning cycle on the recognition effect is analyzed. Figure 7 shows the validation score of the learning cycle set to 20–400 (interval 20). This shows that the validation score increases significantly with the increase of the learning cycle when the learning cycle is less than 200, but increases insignificantly after 200. Identifying efficiency and computing costs, it is reasonable to set a learning cycle of 200.
We compare ADAM-DBN with naïve Bayes (NB), K-nearest neighbor (KNN), support vector machine (SVM) and back propagation neural networks (BPNN) to evaluate the effectiveness of pattern recognition. Here, the Lidstone smoothing parameters of the NB model are α = 2
The pre-processed data sample set is divided into training sets and testing sets by 2:8, as the input of the ADAM-DBM model and the traditional DBN model. The 14 feature sample sets are also divided into training sets and testing sets by 2:8, as the input of the NB, KNN, SVM, and BPNN models. Figure 8 shows the confusion matrix of different methods, where each row represents the actual type, each column represents the predicted type, and the values in the figure represent the proportion of the sample type identified as the sample type in the column. All correct identification results are on the diagonal, and the error is distributing outside the diagonal. In addition to giving the specific recall rate of each defect, the specific error rates of the defect type being misidentified are also given.
As can be seen from Figure 8, the overall identification accuracy of ADAM-DBN model is the highest, where the identification accuracy of the four types of defects has reached more than 95%. Moreover, the accuracy rate of identifying C1 and C2 is more than 98.5%. Compared with the traditional DBN method, ADAM has a higher efficiency of monitoring and optimization of DBN network weight, and the network's mapping of the original signal effectively removes redundant information, so the accuracy of recognition is further improved.
NB, KNN, SVM, BPNN, and DBN method have good overall prediction effectiveness, especially for the identification of defect C2, but performance poor at the identification of C3 and C4. The identify accuracy of C3 and C4 of NB, KNN is less than 90%, and accuracy of SVM, BPNN is less than 92%. This is because C3 and C4 both are situation that part of the insulation is filled with air, so there is a certain similarity between C3 and C4. From Figure 8, we can also see that C3 and C4 are the main types of misidentification of each other, indicating that the characteristics of artificial extraction perform good in C2, but lacks in the identification ability in C3 and C4.
In order to analyze the effect of training data scale on the recognition accuracy, 4000 of the 6400 samples were used as test data set, and the remaining samples were treated as training sets of 400, 800, 1200, 1600, 2000, and 2400, respectively. The results of identification accuracy of NB, KNN, SVM, BPNN, DBN, and ADAM-DBN under six different scales of training data set are shown in Table 1.
From Table 1, it can be seen that with the increase scale of training set, recognition accuracy of KNN, SVM, BPNN, traditional DBN and ADAM-DBN is increasing, but accuracy of NB has not significantly improved. In the case of small training set, the recognition accuracy of DBN is similar to that of other classification algorithms. However, with the increase of the number of training set, the characteristics of the local pulse waveform extracted by the deep learning network are more comprehensive, thus the recognition accuracy of DBN is obviously better than other four methods. At the same time, the modified DBN model supervised fine-tuned by the ADAM algorithm converges faster and the recognition accuracy is higher than other methods on the same training scale.
In this paper, a PD pattern recognition method of DC XLPE cable based on DBN algorithm is proposed to distinguish the different types of defect. Using the PD signal data collected by different artificial defects as input, the pattern recognition effect is analyzed, and the results are shown below:
- The Canny operator is used to pre-process the PD pulse waveform of the XLPE cable. The modified DBN model supervised fine-tuned by the ADAM algorithm is trained to get the pattern recognition result.
- Compared with the NB, KNN, SVM, and BPNN algorithms, ADAM-DBN can unsupervised extract the characteristic information contained in the PD pulse waveforms. And the recognition accuracy of the typical insulation defects in DC XLPE cables is higher than other methods.
- In the experiment, it is found that the traditional classification method based on statistical characteristics performs not so well in the identification of air gap and scratch defects. However, the pattern recognition method based on DBN can effectively characterize the intrinsic relationship between the insulation defect and the PD pulse current waveforms, and have a better recognition effect on all kinds of defects.
- With the increase of training sample size, the recognition accuracy of DC cable based on ADAM-DBN is increased, and the recognition effect is better than that of traditional DBN and other classification methods.
Graph: Figure 1 Structure of DBN.
Graph: Figure 2 Typical insulation defects for XLPE cables.
Graph: Figure 3 PD Experiment and Detection System of DC XLPE Cable.
Graph: Figure 4 Time domain signals of discharge pulse.
Graph: Figure 5 Preprocessed discharge pulse of four defects.
Graph: Figure 6 Relationship between the number of hidden units and the validation score.
Graph: Figure 7 Relationship between learning cycle and the validation score.
Graph: Figure 8 Confusion matrix of insulation recognition results.
Table 1 Recognition accuracy on different scales of training sample sets.
Training Set Recognition Accuracy% NB KNN SVM BPNN DBN ADAM-DBN 400 90.4 93.8 94.6 93.9 92.8 93.8 800 90.1 94.3 95.3 94.5 95.1 95.9 1200 90.4 94.7 95.9 95.2 96.1 96.8 1600 90.5 94.8 96.0 95.3 97.0 98.0 2000 90.6 95.0 96.1 95.4 97.2 98.2 2400 90.5 95.1 96.1 95.6 97.3 98.4
Methodology, Z.L.; software, Y.X.; validation, X.J. All authors have read and agreed to the published version of the manuscript.
This work was in part financially supported by Shanghai Natural Science Foundation (19ZR1426100) to which the authors are indebted.
The authors declare no conflict of interest.
By Zhe Li; Yongpeng Xu and Xiuchen Jiang
Reported by Author; Author; Author