The current understanding of CO2 emission concentrations in hybrid vehicles (HVs) is limited, due to the complexity of the constant changes in their power-train sources. This study aims to address this problem by examining the accuracy, speed and size of traditional and advanced machine learning (ML) models for predicting CO2 emissions in HVs. A new long short-term memory (LSTM)-based model called UWS-LSTM has been developed to overcome the deficiencies of existing models. The dataset collected includes more than 20 parameters, and an extensive input feature optimization has been conducted to determine the most effective parameters. The results indicate that the UWS-LSTM model outperforms traditional ML and artificial neural network (ANN)-based models by achieving 97.5% accuracy. Furthermore, to demonstrate the efficiency of the proposed model, the CO2-concentration predictor has been implemented in a low-powered IoT device embedded in a commercial HV, resulting in rapid predictions with an average latency of 21.64 ms per prediction. The proposed algorithm is fast, accurate and computationally efficient, and it is anticipated that it will make a significant contribution to the field of smart vehicle applications.
Keywords: hybrid vehicles; IoT; CO2; LSTM
Vehicles are a major source of pollution, accounting for a total of 25% of annual CO
Recent developments in machine learning (ML) mechanisms have led to the creation of pollution predictors. However, these have primarily been focused on conventional internal combustion engine (ICE) vehicles. The complexity and constantly changing nature of power-train sources, coupled with several factors involved in determining the CO
In [[
In [[
In [[
In [[
In [[
In [[
In [[
In [[
To sum up, almost all the current literature focuses on prediction of CO
To tackle this problem, we designed, implemented and evaluated a long short-term memory-based (LSTM) model named UWS-LSTM for predicting CO
- Collection of a comprehensive dataset containing operational and emission-related information of a HV in real driving conditions and scenarios.
- Study of the most influential features presented in the dataset for prediction of CO
2 emission concentrations. - Design, implementation, training and validation of an accurate lightweight LSTM-based model (UWS-LSTM) able to perform CO
2 -concentration prediction for HVs. - Comparison of traditional and advanced ML models in terms of accuracy, speed and model size with the proposed UWS-LSTM model.
- Deployment of the designed model in a low-powered IoT device installed within the vehicle to perform real-time on-road CO
2 -concentration prediction.
The rest of the paper is organised as follows. Section 2 elaborates the data collection and cleaning process, and addresses the input feature selection of the models evaluated. Moreover, it describes the definition of the proposed LSTM model and presents the execution environment. Subsequently, Section 3 explains the results found during the evaluation of each of the models evaluated. Section 4 concludes the paper.
This sections explains the main stages addressed to achieve the contributions highlighted in the introduction. Figure 1 depicts the process followed to perform these stages. First, a data collection and cleaning process was carried out to gather the dataset that was used to train and evaluate the ML models. During this process, a study was conducted to investigate the correlation and effect of different input parameters sets on the assessed ML models. As a result, a subset of parameters present in the original dataset was identified as the final dataset and used in subsequent stages of the study. Later, a model creation stage was conducted with the objective of defining, training and validating the ANN-based ML models. Lastly, during the execution stage, an evaluation of each ML model was carried out with various performance indicators when these models were executed in a IoT device installed in a real vehicle. Further descriptions of the tasks involved in the defined stages are elaborated below.
ML is a frequently used method to be applied for classification and regression in myriad of applications. In our study, 5 different ML algorithms were developed and tested to assess their performances when predicting the CO
- Linear regression: Linear regression tries to find the relationship between two variables by fitting a linear equation to observed data. One variable is the explanatory variable, and the other is the dependent variable [[
14 ]]. - Random forest: Random forest regression is a supervised learning algorithm using ensemble learning method to perform regression. Ensemble learning method is a technique that each ML algorithm makes its own individual prediction. These predictions are then averaged to produce a more robust result than a single model [[
15 ]]. - Gradient boosting trees: Gradient boosting decision trees are ensembles of several decision trees which are combined using loss functions to form a strong and effective model. The gradient boosting algorithm uses the gradient descent method to find the optimal point by minimising loss function.
- Artificial neural network (ANN): An ANN is based on a collection of connected nodes inspired by biological neural networks in the brain. The connection between any two neurons has some weight, which determines the effect of one neuron on the other. ANNs can be used to discover nonlinear and complex relationships among input and output variables [[
16 ]]. - Long short-term memory: LSTM is a special form of a recurrent neural network (RNN) that solves the problem of vanishing gradient in vanilla RNNs [[
17 ]]. LSTM models can store information to deal with time-series data due to their memory structure. In the LSTM model, a gating mechanism stores a long sequence of data and uses some information from previous steps to produce the output [[18 ]].
For the entire process of dataset collection, a SprintIR-R 20 (Gas Sensing Solutions Ltd., Cumbernauld, UK) sensor was used for measuring and recording the vehicle's exhaust CO
During the dataset collection process, the sensor was installed in the rear end of a vehicle and connected to the vehicle's exhaust pipe. Furthermore, a set of tools were used to separate impurities and water vapour from the CO
The collection of the onboard vehicle status parameters was carried out with custom software executed on a laptop that was connected to the vehicle's CAN bus via its OBD2 port. The software was written in Python and performs UDS-like (Unified Diagnostic Services) [[
The set of parameters acquired from the vehicle's CAN bus comprises the following 21 parameters: accelerator position (%), vehicle speed (MPH), external atmospheric pressure (psi), engine load (%), coolant temperature (°C), acceleration (m/s
- Drive mode: indicates whether the power train source is hybrid or electric.
- EV mode status: in case the drive mode is electric, it indicates whether the electric mode is normal or "city", which limits the power capacity up to 53 kW.
- Engine mode: indicates the status of the engine, such as (
1 ) stopped, (2 ) stopping, (3 ) starting or (4 ) started.
Specific input features selection was conducted to identify a subset of input features that are most pertinent to the output variable and also to eliminate unnecessary information in the prediction process. To perform the input selection, the significance of the input data and the correlation amongst features should be estimated. Hence, to identify the most relevant parameters for this use-case, the correlation coefficient technique and some traditional ML algorithms were used for different parameters of the dataset.
The first attempt to identify the most useful parameters was to calculate their correlation coefficients. Firstly, a correlation matrix was created with the entire set of parameters described in Section 2.3. The technique implemented was the Spearman-based correlation matrix, which provides a correlation coefficient based on the statistical dependence of ranking between two variables and covers non-linear relationships. The parameters with a high correlation score among them were eliminated, and the ones with a high correlation with the CO
To further evaluate the impacts of the remaining input parameters, 255 different combinations of input values (the combination for 8 input values) were used to train and test each ML model to assess the adjusted R
Table 3 illustrates the top 4 combinations of input parameters that showed the best result for each ML algorithm tested. As seen, for both RF and GBR, the set of parameters that achieved the highest scores were: acceleration, hybrid battery SOC, vehicle speed, engine mode, coolant temperature and engine speed, leaving behind mass air flow and engine exhaust flow rate. However, LR showed that using all the values yielded the best score. In this case, as the RF and GBR obtained substantially higher accuracy than LR, the combinations of input parameters indicated by the best GBR and RF scores were chosen as candidates for the set of most influential parameters. As a result, two different set of input parameters were defined and used in the following stages of this research:
- Input set 1. Features 8 input parameters: acceleration, hybrid battery SOC, vehicle speed, engine mode, coolant temperature, engine speed, mass air flow and engine exhaust flow rate.
- Input set 2. Features 6 input parameters: acceleration, hybrid battery SOC, vehicle speed, engine mode, coolant temperature and engine speed.
A total of 70,683 samples were collected from 235 min of driving. Three heterogeneous road scenarios are included in this dataset: motorway roads, urban roads and intercity roads; all of them were recorded at different times of the day. The classification of the samples in each road was done manually based on the GPS coordinates of the vehicle at each moment. As is common, 80% of the samples were employed for the purpose of training (56,546 samples in total), and the remaining 20% (14,137 samples in total) were used for validation and testing (10% and 10% respectively).
The distribution of CO
The high frequency of switch-overs recorded during the drives carried out, which is very usual and normal behaviour in HV, and a non-constant CO
For training and validation, the complete dataset was standardised to prevent inputs with higher values from dominating those with the smaller values. To this end, formulae described by Equation (
(
The proposed UWS-LSTM comprises three primary components: the first component is the input layer, which includes the most effective input parameters mentioned in Section 2.4.2. The second component is the hidden layers comprising two LSTM layers with 512 and 256 neurons in each LSTM cell. In this regard, a hierarchical structure of stacked LSTM layers has been shown to outperform single-layer LSTM models [[
One of the advantages of using an LSTM model in comparison with the ANN model is that the former can be parameterized in terms of the amount of information that is taken into consideration from previous states. This term is often referred to as the lookback window. This parameter determines how many previous timesteps are considered to perform the current prediction. In order to figure out the influence of this parameter in the accuracy of the LSTM model, a process of model training and validation was conducted with a set of different lookback windows to compare the obtained scores. Further results can be observed in Section 3.1.1.
This subsection gathers the information regarding the configurations for the ML models studied in this work. Table 4 reflects the hyperparameter configurations. It is worth mentioning that for optimising the hyperparameters defined in the ANN model, a series of Bayesian optimisation iterations were conducted using keras-tuner.
The complete prediction system was deployed in a real operating vehicle and tested on public roads to validate its performance under realistic conditions. The vehicle used was a Toyota Prius Plug-in Hybrid (2019). This vehicle features a 4-stroke, 4-cylinder 1798 cc gasoline engine that can output 90 kW of power. The maximum torque achieved is 142 Nm at 3600 RPM. As per the emissions, the vehicle is categorised under the Euro-class scheme as a Euro 6 DG type; according to official manufacturers information, the vehicle emits 28 g/km of CO
The computing platform chosen was a Nvidia Jetson Xavier. This board is an off-the-shelf, low-consumption Linux-based system with a 512-core Volta GPU with Tensor Cores which runs at more than 21 tera operations per second (TOPS) and has 32 GB of memory. Different power consumption modes are supported: 10 W, 15 W and 30 W. Its portability and its power capability made it suitable for use cases to deploy our CO
All the ANN-based algorithms were implemented and executed on TensorFlow 2. TensorFlow [[
In this section, we present and analyse the results obtained from the evaluations of the ML models outlined in previous sections of the manuscript. The results are presented in two forms: quantitative and qualitative. The quantitative results consist of a set of numerical measurements, and the qualitative results pertain to the implementation and performance of the CO
The quantitative accuracy of the predictions was measured by calculating the errors given by adjusted R
(
(
(
(
As previously mentioned in Section 2.6, a key hyperparameter to be defined for LSTM-based models is the lookback window. Table 5 presents the different results in terms of accuracy after training and validating the UWS-LSTM model with different lookback window sizes. In addition, we presented in Section 2.4.2 that two particular input parameters yielded different scores depending on the ML model used. For this reason, these input parameters were part of the analysis to determine the best LSTM configuration.
As observed in the results, configurations with smaller lookback window sizes, such as 1, 2 or 4, performed inferiorly. However, there was a significant increase in accuracy as the window size increased: an improvement of approximately 25% when using both input parameter sets, when comparing a window size of 8 to 16. Notably, the highest accuracy was achieved with a window size of 32 and input parameter set 1: an adjusted R
Nonetheless, it is also known that higher lookback windows demand higher computational resources. Since the aim of this research was the deployment of the developed model in a low-powered device, no bigger window sizes were considered for this study.
Table 6 shows the accuracy obtained for each ML model. It is very important to clarify that, due to the fact that LSTM-based models are naturally designed to be trained (and executed) following a time-based order, for the sake of a fair comparison among all the models, the other 4 ML algorithms were trained following the same procedure, i.e., without shuffling the testing dataset. Figure 6 shows the prediction results of the models. The figures show the real values against the predicted values in ppm. In addition, the results are colour-coded based on the differences between these two values.
As evident in Table 6, the performance of the LR model was the lowest among all the ML models evaluated, with scores of 0.4214, 0.6566, 0.6021 and 0.7759 for adjusted R
Ultimately, the proposed UWS-LSTM algorithm demonstrated superior performance in comparison to the other evaluated methods, as evidenced by its adjusted R
This subsection focuses on the experiments carried out after deploying and applying our predictive model in TensorFlow in the Nvidia Jetson board described in Section 2.8. In addition to the proposed UWS-LSTM model, LR, RF, GBR and ANN were studied to compare the different latency levels and assess the trade-offs between speed and accuracy. All these results were obtained by setting the Nvidia Jetson Xavier on different power modes: 10 W, 15 W and 30 W.
The results in Figure 7 are presented in milliseconds, which shows the cumulative average inference time for the first 1000 iterations. The cumulative average inference time was calculated following Equation (
(
First the performance levels of traditional ML algorithms, including LR, GBR and RF, were analysed and compared. The results, as depicted in Figure 7a–c, indicate that the LR algorithm performed with the lowest average cumulative latency of 0.16 ms per iteration. GBR exhibited a slightly slower performance, with an average execution time of 1.43 ms per iteration under a 30 W power consumption. On the other hand, RF algorithm demonstrated the highest average latency of 14.45 ms among the three traditional ML algorithms. Furthermore, ANN and LSTM models, as presented in Figure 7d,e, respectively, both exhibited slower average performance when operating at 30 W. Specifically, the ANN-based model recorded an average execution time of 5.72 ms, whereas the LSTM-based model took 21.64 ms on average. These results suggest that traditional ML algorithms generally exhibit faster performance compared to ANN and LSTM models. However, as seen in previous sections, there is a trade-off between speed and accuracy, which is especially notable with the UWS-LSTM model.
Additionally, as observed in every graph, there was a significant improvement in terms of speed when the Nvidia Jetson board was configured to operate at 15 W in comparison to 10 W. Oppositely, changing from 15 to 30 W did not reveal a remarkable enhancement in terms of latency.
In an IoT setting, the size of a machine learning model is a crucial consideration. Due to the limited storage and computational resources of IoT devices, models with smaller sizes are typically more favourable, as they can be more easily exported and deployed on these devices. In this context, the UWS-LSTM model, with a size of 7.7 MB, can be considered relatively lightweight and well-suited for IoT applications. As depicted in Figure 8, among the five models presented, only the LR model had a smaller size than the UWS-LSTM model. However, as previously established through the analysis of other performance metrics, the LR model dud not present an appropriate trade-off between model size and accuracy. Thus, the UWS-LSTM model emerged as the most promising candidate for deployment on IoT devices.
The UWS-LSTM model was instantiated and installed within a real scenario to prove the potential of its use under realistic scenarios. The Nvidia Jetson was set up in the Toyota Prius car described in Section 2.8. The installation can be seen in Figure 9a, where the display shows the real-time execution and CO
During the on-board evaluation, the real-time predictions were shown on a display. However, IoT devices represent an ideal platform for remote monitoring, thanks to their easy integration with communication protocols over wireless networks, leveraging an incipient solution for remote ML-based emissions monitoring.
In this paper, we have presented the design, implementation, training and validation of a LSTM-based ML model for prediction of CO
To determine the relevance of the gathered data, we performed an input feature selection by calculating the correlation score among the dataset parameters and evaluating 765 different combinations of training with different input features in three traditional ML algorithms. This resulted in two sets of parameters, one with six parameters and another with eight parameters, which was better for each different algorithm, showcasing that no specific input parameter set can be foreseen as optimal for every ML model. This input feature selection procedure allowed identifying the most relevant parameters for predicting emissions and eliminating irrelevant or redundant data, which proved to improve the performance in the assessed models.
Moreover, we have designed and evaluated the proposed ML model, UWS-LSTM, for predicting CO
Finally, we have implemented the UWS-LSTM model in a low-powered IoT device installed in the same vehicle where we gathered the data (see Figure 9). An evaluation of its performance with three different power settings was conducted as recorded to measure the speed. As observed, the UWS-LSTM reached a minimum average execution time of 21.64 ms, which was considerably more than the second best ML algorithm (GBR with 1.43 ms). Nonetheless, as for the model sizes, the proposed UWS-LSTM was the second most lightweight model, being 7.7 MB; the other three most accurate models were 150.1, 151.4 and 624.5 MB, respectively. Overall, the proposed UWS-LSTM seems to better fit the performance and weight requirements for being implemented in low-resource environments.
In conclusion, the work presented in this study would make an important contribution to the field of HV emission concentration prediction. The large collected dataset, the effective input feature selection and the performance of the UWS-LSTM model make it a valuable resource for researchers in the area of ML-based pollution prediction, which can greatly help the development of new cost-effective emission-aware driving strategies. Moreover, thanks to the use of IoT devices, this solution can potentially facilitate the transmission of CO
Despite the provided results proving the capacity of the model to accurately predict the concentrations with a decent extent of success, we are aware of the shortcomings in execution time. Therefore, further research will be conducted towards reducing the execution time for the proposed model.
This paper presented the design and implementation of an LSTM-based ML model for predicting a CO
Graph: Figure 1 Workflow of the complete process from when the dataset was collected from the vehicle to the final execution, including designing and training of the ML model.
Graph: Figure 2 Data collection sequence diagram. Notice that after the acquisition of every ECU parameter and the CO2 concentration, the received messages have to be re-scaled and transformed from hexadecimal to decimal notation.
Graph: Figure 3 Correlation matrix composed of the final input values selected.
Graph: Figure 4 Histogram of CO2 concentration (ppm) present in the dataset.
Graph: Figure 5 Architecture of proposed LSTM-based algorithm.
Graph: Figure 6 Prediction results. The colour-code is based on the difference between the original data and the predicted data. (a) Very dispersed predicted values with an R 2 of 42%. (b) Better prediction in ppm edges. Still high dispersion. (c) GBR still failing in predicting the CO2 in switch-overs. (d) The ANN performed very poorly on power-train switch-overs, with an accuracy of 46%. (e) LSTM narrows all predictions to the ground truth.
Graph: Figure 7 Performance evaluation. (a) Outstanding results in speed. (b) Good results at 15–30 W. (c) Less than 10 ms for each scenario. (d) Good for 15–30 W. (e) Achieved real-time for 15–30 W.
Graph: Figure 8 Comparison of the ML models size.
Graph: Figure 9 Vehicle onboard deployment. (a) Setup. (b) Prediction against real recorded values.
Graph: sensors-23-01350-g009b.tif
Table 1 Comparison of prediction techniques for vehicle CO
Ref. Prediction Vehicle/ Fuel Algorithm Framework Exec. Accuracy Exe. Model [ CO2 conc. ng ng GBR ng ng 91 ng ng [ CO2 emission Passenger/ Diesel GBR Scikit-learn ng 99 ng ng [ CO2 emission Passenger/ Gasoline XGBoost Scikit-learn ng 89.8 ng ng [ CO2 emission Passenger/ Gasoline LSTM ng PC 9.30 ng ng [ CO2 emission Passenger/ Gasoline GPR ng ng 69 ng ng [ CO2 Passenger/ Gasoline LR ng ng 95.75 ng ng [ CO2 conc. ng/ICE Diesel PA-LSTM ng PC 94.6 ng ng [ CO2 conc. Construction/ Diesel RF ng ng 94 ng ng [ CO2 Passenger/ Diesel ANN TensorFlow ng 0.5 ng ng TP CO2 conc. Passenger/ Gasoline LSTM TensorFlow Nvidia 97.5 21.64 7.7
Table 2 Statistical values for each of the input features selected in the final dataset.
ID Name Unit Mean Min. Max. 1 Acceleration m/s2 0.011 −7.058 4.634 2 Hybrid battery SOC % 16.63 10.58 78.55 3 Vehicle speed MPH 32.61 0 79.535 4 Engine mode n/a n/a n/a n/a 5 Coolant temperature °C 83.24 23 93 6 Engine speed RPM 730.734 0 4512 7 Mass air flow g/min 424.836 25.2 3753.6 8 Engine exhaust flow rate kg/h 26.074 0 242.2
Table 3 Top 4 most accurate combinations of different input parameters with three traditional ML algorithms. The black dots indicate that the parameter was included in the input parameter for that combination, whereas the white dots indicate the opposite.
ML Input Parameters Adjusted Acceleration Hybrid Vehicle Engine Coolant Engine Mass Engine LR ● ● ● ● ● ● ● ○ 0.518 ● ● ● ● ● ● ● ● 0.518 ● ● ● ● ○ ● ● ○ 0.517 ● ● ● ● ○ ● ● ● 0.517 RF ● ● ● ● ● ● ○ ○ 0.91 ● ● ● ○ ● ● ● ○ 0.91 ● ● ● ● ● ● ● ○ 0.909 ● ● ● ● ● ○ ● ○ 0.908 GBR ● ● ● ● ● ● ○ ○ 0.916 ● ● ● ○ ● ● ○ ○ 0.913 ● ● ● ● ● ○ ○ ● 0.909 ● ● ● ● ● ○ ● ○ 0.909
Table 4 Final hyperparameter configuration for each ML model.
Algorithm Input No. Loss Max. Min. Max. Subsample Learning Epochs Batch Optimiser Lookback LR 1 n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a n/a RF 2 800 MSE None 2 None n/a n/a n/a n/a n/a n/a GBR 2 800 MSE 12 10 n/a 0.8 0.08 n/a n/a n/a n/a ANN 1 n/a MSE n/a n/a n/a n/a 1000 32 Adam n/a UWS-LSTM 1 n/a MSE n/a n/a n/a n/a 100 32 Adam 32
Table 5 Accuracy score for each LSTM network configuration.
(a) Input Parameters Set 1 (b) Input Parameters Set 2 1 0.5627 1 0.5585 2 0.5015 2 0.4895 4 0.4738 4 0.4446 8 0.6036 8 0.5237 16 0.8389 16 0.7901 32 0.975 32 0.9702
Table 6 Accuracy scores obtained with the complete test dataset. Note that the scores shown refer to standardised data (see Equation (
Algorithm Adjusted R MAE MSE RMSE LR 0.4214 0.6566 0.6021 0.7759 RF 0.4645 0.5507 0.5572 0.7465 GBR 0.4597 0.5555 0.5622 0.7498 ANN 0.4639 0.5346 0.5577 0.7468 UWS-LSTM 0.975 0.1135 0.0261 0.1616
Conceptualisation, D.T.-G., G.G. and I.M.-A.; methodology, D.T.-G., G.G. and I.M.-A.; software, D.T.-G.; validation, D.T.-G.; I.M.-A. formal analysis, D.T.-G.; investigation, D.T.-G., G.G. and I.M.-A.; data curation, D.T.-G., G.G. and I.M.-A.; writing—original draft, D.T.-G., G.G. and I.M.-A.; writing—review and editing, Q.W. and J.M.A.-C.; visualization, D.T.-G.; project administration, D.T.-G.; funding acquisition, Q.W. and J.M.A.-C. All authors have read and agreed to the published version of the manuscript.
Not applicable.
Not applicable.
Data available on request due to privacy restrictions.
The authors declare no conflict of interest.
The following abbreviations are used in this manuscript:
ANN Artificial Neural Network CAN Controlled Area Network cc cubic centimeters CNN Convolutional Neural Network CO2 Carbon Dioxide CPU Central Processing Unit CUDA Compute Unified Device Architecture ECU Electronic Control Unit GBR Gradient Boosting Regression GPS Global Positioning System GPU Graphics Processing Unit HV(s) Hybrid Vehicle(s) ICE Internal Combustion Engine IoT Internet of Things LR Linear Regression LSTM Long Short-Term Memory MAE Mean Absolute Error ML Machine Learning MLP Multi Layer Perceptron MPH Miles per Hour MSE Mean Square Error NDIR Non Dispersive Infra Red NOx Nitrogen Oxide PEMS Portable Emissions Monitoring System ppm parts per million psi pounds per square inch RF Random Forest RMSE Root Mean Square Error RNN Recurrent Neural Network RPM Revolutions per Minute SOC State of Charge SVR Support Vector Regression TOPS Tera Operations per Second UDS Unified Diagnostic Services W Watts WLTP Worldwide Harmonised Light Vehicle Test Procedure XGBR Xtreme Gradient Boosting Regression
Video of instantaneous CO
By David Tena-Gago; Gelayol Golcarenarenji; Ignacio Martinez-Alpiste; Qi Wang and Jose M. Alcaraz-Calero
Reported by Author; Author; Author; Author; Author