LSTM and GRU model analysis for time series forecasting

S.V. Kozlov, S.A. Sedenkov

Abstract


 The article presents an analysis of the application of recurrent neural networks as a tool for predicting time series. General principles of operation of recurrent neural networks are briefly described. The advantages of their use in comparison with standard neural networks and convolutional networks are described. Application areas of different types of recurrent neural network architecture are considered. The algorithm of functioning of recurrent neural networks was analyzed. The main class is defined in the description of the algorithm, the software implementation of its functions is given. Particular attention is paid to the matrix form of the parameters when executing the algorithm. The main part of the work is devoted to a comparative analysis of the model of a long chain of short-term memory elements and the model of controlled recurrent blocks. The article briefly describes the history of their development, the principle of operation of each model. The possibilities of these models in solving problems of constructing time series are considered. Disclosed is the essence of formulas, with the help of which neural networks LSTM and GRU perform calculations. The program code developed by the authors for each of the models is characterized. IBM stock price forecasting was chosen to analyze the operation of algorithms. The data obtained during the experiment are shown in the graphs. At the end of the work, their comparative analysis is given. The relevance of the article is due to the effectiveness of the implementation of methods for recurrent analysis of time series data using neural networks.

Full Text:

PDF (Russian)

References


Antonov V. V., Pal'chevskij E. V., Enikeev R. R. Prognozirovanie na osnove iskusstvennoj nejronnoj seti vtorogo pokolenija dlja podderzhki prinjatija reshenij v osobo znachimyh situacijah // Programmnye produkty i sistemy. – 2022. – # 3. – S. 384-395.

Kozlov S. V. Ispol'zovanie funkcional'nyh vozmozhnostej informacionnyh sistem v proizvodstvennoj sfere // JeNERGETIKA, INFORMATIKA, INNOVACII – 2017 (jelektrojenergetika, jelektrotehnika i teplojenergetika, matematicheskoe modelirovanie i informacionnye tehnologii v proizvodstve). Sbornik trudov VII-oj Mezhdunarodnoj nauchno-tehnicheskoj konferencii. – 2017. – V 3 t. T 1. – S. 298-301.

Fahrutdinova A. Z. Metodologicheskie problemy sovremennoj prognostiki // Omskie nauchnye chtenija. Materialy Vserossijskoj nauchno-prakticheskoj konferencii. – 2017. – S. 851-853.

Kozlov S. V., Suin I. A. O nekotoryh aspektah primenenija invariantnyh metodov funkcional'nogo analiza dannyh v razlichnyh predmetnyh oblastjah // Sistemy komp'juternoj matematiki i ih prilozhenija. – 2019. – # 20-1. – S. 199-205.

Dubenko Ju. V., Dyshkant E. E. Nejrosetevoj algoritm vybora metodov dlja prognozirovanija vremennyh rjadov // Vestnik Astrahanskogo gosudarstvennogo tehnicheskogo universiteta. Serija: Upravlenie, vychislitel'naja tehnika i informatika. – 2019. # 1. S. 51-60.

Kozlov S. V. Ispol'zovanie sootvetstvija Galua kak invarianta otbora kontenta pri proektirovanii informacionnyh sistem // Sovremennye informacionnye tehnologii i IT-obrazovanie. – 2015. – T. 2. # 11. – S. 220-225.

Alymova E. V. Sovmestnoe primenenie modeli linejnoj regressii i nejronnoj seti v zadache predskazanija trenda kotirovok kriptovaljuty Bitcoin // Inzhenernyj vestnik Dona. – 2020. – # 10 (70). – S. 90-96.

Senotova S. A. Sravnitel'nyj analiz metodov approksimacii s pomoshh'ju regressionnyh zavisimostej i nejronnyh setej dlja linejnyh modelej // Sbornik nauchnyh trudov Angarskogo gosudarstvennogo tehnicheskogo universiteta. – 2021. – T. 1. # 18. – S. 31-35.

Solov'eva E. B. Rekurrentnye nejronnye seti v kachestve modelej nelinejnyh dinamicheskih sistem // Cifrovaja obrabotka signalov. – 2018. – # 1. – S. 18-27.

Dubolazov V. A., Somov A. G. Sozdanie nelinejnyh approksimacij sovremennyh jekonomicheskih modelej metodom nejronnyh setej // Innovacionnye klastery cifrovoj jekonomiki: drajvery razvitija. Trudy nauchno-prakticheskoj konferencii s mezhdunarodnym uchastiem. Pod redakciej A.V. Babkina. – 2018. – S. 481-487.

Andreev K. V., Bykov A. A., Kiseleva O. M. Matematicheskaja model' prediktivnogo kodirovanija radiotehnicheskih signalov, osnovannaja na algoritme izmenjajushhegosja shaga kodirovanija // Sovremennye naukoemkie tehnologii. 2020. – # 11-2. – S. 261-267.

Andrievskaja N. V. Identifikacija nelinejnoj modeli s ispol'zovaniem modelej nechetkoj logiki i iskusstvennyh nejronnyh setej // Nejrokomp'jutery: razrabotka, primenenie. – 2017. – # 6. – S. 3-8.

Zotkina A. A. Rekurrentnye nejronnye seti kak algoritm posledovatel'nosti dannyh // Sovremennye informacionnye tehnologii. – 2022. – # 35 (35). – S. 24-26.

Kurov A. S., Nikolaeva I. V. Rekurrentnye nejronnye seti kak instrument prognozirovanija vremennyh rjadov // Informacionnoe obshhestvo: sovremennoe sostojanie i perspektivy razvitija. Sbornik materialov XII mezhdunarodnogo foruma. – 2019. – S. 272-275.

Borisenkova A. V., Kozlov S. V. Ispol'zovanie metoda kaskadov Haara pri raspoznavanii obrazov na izobrazhenijah // Razvitie nauchno-tehnicheskogo tvorchestva detej i molodezhi: Sbornik materialov III Vserossijskoj nauchno-prakticheskoj konferencii s mezhdunarodnym uchastiem. – 2019. – S. 28-33.

Qi-Qiao He, Cuiyu Wu, Yain-Whar Si LSTM with particle Swam optimization for sales forecasting. Electronic Commerce Research and Applications, Volume 51, Jan-Feb 2022. https://doi.org/10.1016/j.elerap.2022.101118

Moshkarova L. A., Tel'minov O. A. Metody izvlechenija akusticheskih priznakov v zadache raspoznavanija rechi rekurrentnymi nejronnymi setjami s dolgoj kratkosrochnoj pamjat'ju // Nanoindustrija. – 2020. – T. 13. – # S5-3 (102). – S. 838-841.

Fedotov D. V., Verholjak O. V., Karpov A. A. Kontekstnoe nepreryvnoe raspoznavanie jemocij v russkoj rechi s ispol'zovaniem rekurrentnyh nejronnyh setej // Analiz razgovornoj russkoj rechi (ARz-2019). Trudy vos'mogo mezhdisciplinarnogo seminara. – 2019. – S. 96-99.

Bagaev I. I. Analiz ponjatij nejronnaja set' i svertochnaja nejronnaja set', obuchenie svertochnoj nejroseti pri pomoshhi modulja Tensorflow // Matematicheskoe i programmnoe obespechenie sistem v promyshlennoj i social'noj sferah. – 2020. – T. 8. # 1. – S. 15-22.

Zaharov V. N., Munerman V. I. Parallel'nyj algoritm umnozhenija mnogomernyh matric // Sovremennye informacionnye tehnologii i IT-obrazovanie. – 2015. – T. 11. # 2. – S. 384-390.

Bolotova Ju. A., Fedotova L. C., Spicyn V. G. Algoritm detektirovanija oblastej lic i ruk na izobrazhenii na osnove metoda Violy-Dzhonsa i algoritma cvetovoj segmentacii // Fundamental'nye issledovanija. – 2014. – # 11 - 10. – S. 2130 - 2134.

Averkin A. N., Sobolev S. V., Voroncov A. O. Sravnenie razlichnyh tehnik analiza jemocij dlja reshenija zadachi vizualizacii indeksa nastroenija // Mjagkie izmerenija i vychislenija. – 2019. – #11. (24). – S. 30-34.

Kozlov S. V., Svetlakov A. V. O LL(1)-grammatikah, algoritmah na nih i metodah ih analiza v programmirovanii // International Journal of Open Information Technologies. – 2022. T. 10. # 3. – S. 30-38.

Tasarruf Bashir, Chen Haoyong, Muhammad Faizan Tahir, Zhu Liqiang Short term electricity load forecasting using hybrid prophet-LSTM model optimized by BPNN. Energy Reports, Volume 8, November 2022, Pp. 1678-1686, Energy Reports. https://doi.org/10.1016/j.egyr.2021.12.067

Jiaqi Qin, Yi Zhang, Shixiong Fan, Xiaonan Hu, Yongqiang Huang, Zexin Lu, Yan Liu Multi-task short-term reactive and active load forecasting method based on attention-LSTM model. International Journal of Electrical Power & Energy Systems, Vol. 135, February 2022. https://doi.org/10.1016/j.ijepes.2021.107517

Shiva Nosouhian, Fereshteh Nosouhian, Abbas Kazemi Khoshouei A review of recurrent neural network architecture for sequence learning: comparison between LSTM and GRU. Preprints 2021, 2021070252 https://doi.org/10.20944/preprints202107.0252.v1

Xu G., Peng Sh., Li Ch., Chen X. Synergistic evolution of China’s green economy and digital economy based on LSTM-GM and grey absolute correlation // Sustainability. 2023. Vol. 15. # 19. P. 14156.

Ezat Ahmadzadeh, Hyunil Kim, Ongee Jeong, Namki Kim, Inkyu Moon A deep bidirectional lstm-gru network model for automated ciphertext classification. IEEE Access, Vol. 10. P. 3228-3237.

Savchenko V. V. Metod avtoregressionnogo modelirovanija rechevogo signala s ispol'zovaniem ogibajushhej periodogrammy Shustera v kachestve opornogo spektral'nogo obrazca // Radiotehnika i jelektronika. – 2023. – T. 68. # 2. – S. 138-145.

Ya Gao, Rong Wang, Enmin Zhou Stock prediction based on optimized LSTM and GRU models. Hindawi, Scientific Programming, Volume 2021. https://doi.org/10.1155/2021/4055281


Refbacks

  • There are currently no refbacks.


Abava  Кибербезопасность IT Congress 2024

ISSN: 2307-8162