Sains Malaysiana 51(8)(2022): 2645-2654

http://doi.org/10.17576/jsm-2022-5108-23

Performance of Levenberg-Marquardt Neural Network Algorithm in Air Quality Forecasting

(Prestasi Algoritma Rangkaian Neuron Levenberg-Marquardt dalam Ramalan Kualiti Udara)

CHO KAR MUN¹, NUR HAIZUM ABD RAHMAN^1,* & ISZUANIE SYAFIDZA CHE ILIAS²

¹Department of Mathematics and Statistics, Faculty of Science, Universiti Putra Malaysia 43400 UPM Serdang, Selangor Darul Ehsan, Malaysia

²Institute for Mathematical Research, Universiti Putra Malaysia, 43400 UPM Serdang, Selangor Darul Ehsan, Malaysia

Received: 2 July 2021/Accepted: 30 January 2022

Abstract

Levenberg-Marquardt algorithm and conjugate gradient method are frequently used for optimization in multi-layer perceptron (MLP). However, both algorithms have mixed conclusions in optimizing MLP in time series forecasting. This study uses autoregressive integrated moving average (ARIMA) and MLP with both Levenberg-Marquardt algorithm and conjugate gradient method. These methods were used to predict the Air Pollutant Index (API) in Malaysia's central region where represent urban and residential areas. The performances were discussed and compared using the mean square error (MSE) and mean absolute percentage error (MAPE). The result shows that MLP models have outperformed ARIMA models where MLP with Levenberg-Marquardt algorithm outperformed the conjugate gradient method.

Keywords: Algorithm; ARIMA; artificial neural network; forecasting; multi-layer perceptron

Abstrak

Algoritma Levenberg-Marquardt dan kaedah kecerunan konjugat sering digunakan untuk pengoptimuman dalam perceptron pelbagai lapisan (MLP). Walau bagaimanapun, kedua-dua algoritma mempunyai kesimpulan yang berbeza dalam mengoptimumkan ramalan siri masa menggunakan MLP. Kajian ini menggunakan purata bergerak bersepadu autoregresif (ARIMA) dan MLP dengan kedua-dua algoritma Levenberg-Marquardt dan kaedah kecerunan konjugat. Kaedah ini digunakan untuk meramalkan Indeks Pencemaran Udara (IPU) di wilayah tengah Malaysia yang mewakili kawasan bandar dan kediaman. Prestasi dibincang dan dibandingkan dengan menggunakan ralat kuasa dua min (MSE) dan ralat peratusan mutlak (MAPE). Hasilnya menunjukkan bahawa model MLP telah mengatasi model ARIMA dengan MLP dan algoritma Levenberg-Marquardt mengatasi kaedah kecerunan konjugat.

Kata kunci: Algoritma; ARIMA; perceptron pelbagai lapisan; ramalan; rangkaian neuron tiruan

References

An, N.H. & Anh, D.T. 2015. Comparison of strategies for multi-step-ahead prediction of time series using neural network. 2015 International Conference on Advanced Computing and Applications (ACOMP). pp. 142-149.

Chen, X., Wang, G., Zhou, W., Chang, S. & Sun, S. 2006. Efficient Sigmoid function for neural networks based FPGA design. International Conference on Intelligent Computing. pp. 672-677.

De Cosmi, V., Mazzocchi, A., Milani, G.P., Calderini, E., Scaglioni, S., Bettocchi, S., D’Oria, V., Langer, T., Spolidoro, G.C.I., Leone, L., Battezzati, A., Bertoli, S., Leone, A., De Amicis, R.S., Foppiani, A., Agostoni, C. & Grossi, E. 2020. Prediction of resting energy expenditure in children: May artificial neural networks improve our accuracy? Journal of Clinical Medicine 9(4): 1026.

Dong, G., Fataliyev, K. & Wang, L. 2013. One-Step and multi-step ahead stock prediction using backpropagation neural networks. 2013 9th International Conference on Information, Communications & Signal Processing. pp. 1-5.

Evans, N.J. 2019. Assessing the practical differences between model selection methods in inferences about choice response time tasks. Psychonomic Bulletin & Review 26(4): 1070-1098.

Jain, A.K., Mao, J. & Mohiuddin, K.M. 1996. Artificial Neural Networks: A tutorial. Computer 29(3): 31-44.

Kaastra, I. & Boyd, M. 1996. Designing a Neural Network for forecasting financial and economic time series. Neurocomputing 10(3): 215-236.

Kavzoglu, T. 1999. Determining optimum structure for Artificial Neural Networks. Proceedings of the 25^th Annual Technical Conference and Exhibition of the Remote Sensing Society, Cardiff, UK. 8-10 September. pp. 675-682.

Kermani, B.G., Schiffman, S.S. & Nagle, H.T. 2005. Performance of the Levenberg–Marquardt neural network training method in electronic nose applications. Sensors and Actuators B: Chemical 110(1): 13-22.

Mahmoud Khaki, Ismail Yusoff, Nur Fadilah Islami & Nur Hayati Hussin. 2016. Artificial Neural network technique for modeling of groundwater level in Langat Basin, Malaysia. Sains Malaysiana 45(1): 19-28.

Lodwich, A., Rangoni, Y. & Breuel, T. 2009. Evaluation of robustness and performance of early stopping rules with multi layer perceptrons. 2009 International Joint Conference on Neural Networks. pp. 1877-1884.

Nurulilyana Sansuddin, Nor Azam Ramli, Ahmad Shukri Yahaya, Noor Faizah Fitri Md Yusof, Nurul Adyani Ghazali & Wesam Ahmed Al Madhoun. 2011. Statistical analysis of PM₁₀ concentrations at different locations in Malaysia. Environmental Monitoring and Assessment 180(1-4): 573-588.

Saratha Sathasivam, Mohd. Asyraf Mansor, Ahmad Ismail, Siti Jamaludin, Mohd Kasihmuddin, & Mustafa Mamat. 2020. Novel random k satisfiability for k ≤ 2 in Hopfield neural network. Sains Malaysiana 49(11): 2847-2857.

Sathya, R. & Annamma Abraham. 2013. Comparison of supervised and unsupervised learning algorithms for pattern classification. International Journal of Advanced Research in Artificial Intelligence 2 (February). https://dx.doi.org/10.14569/IJARAI.2013.020206

Sulafa Hag Elsafi. 2014. Artificial Neural Networks (ANNs) for flood forecasting at Dongola Station in the River Nile, Sudan. Alexandria Engineering Journal 53(3): 655-662.

Sheela Tiwari, Ram Naresh & Rahul Jha. 2013. Comparative study of backpropagation algorithms in neural network based identification of power system. International Journal of Computer Science and Information Technology 5(August): 93-107.

Mina Torabi & Mohammad-Mehdi Hosseini. 2018. A new descent algorithm using the three-step discretization method for solving unconstrained optimization problems. Mathematics 6 (April): 63.

Yu, H. & Wilamowski, B.M. 2011. Lavenberg-Marquardt Training. In The Industrial Electronics Handbook: Intelligent Systems, 2nd ed. Chapter 12, edited by Wilamowski, B.M. & Irwin, J.D. Boca Raton: CRC Press. pp. 12-1 to 12-15.

Zafer Cömert & Adnan Fatih Kocamaz. 2017. A Study of artificial neural network training algorithms for classification of cardiotocography signals. Bitlis Eren University Journal of Science and Technology 7(2): 93-103.

Zhang, G.P. 2012. Neural networks for time-series forecasting BT. In Handbook of Natural Computing, edited by Rozenberg, G., Bäck, T. & Kok, J.N. Berlin, Heidelberg: Springer Berlin Heidelberg. pp. 461-477.

Zhang, G., Eddy Patuwo, B. & Hu. M.Y. 1998. Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting 14(1): 35-62.

Zhu, L., Lim, C. & Zhang, J. 2020. Prediction of risk: Decoding the serial dependence of stock return volatility with Copula. Journal of Hospitality & Tourism Research 45(1): 6-27.

^*Corresponding author; email: nurhaizum_ar@upm.edu.my