Research on the Application of Deep Learning Models in Time Series Data
DOI:
https://doi.org/10.54097/fk1ztr70Keywords:
Time Series Forecasting, Transformer Model, Hyperparameter Optimization.Abstract
In today's digital age, In the digital age, the importance of time-series data is increasingly evident across finance, industry, IoT, and healthcare, with its volume growing rapidly. According to IDC projections, global data volume will reach 175 ZB by 2025, with approximately 79.4 ZB originating from IoT devices—much of it existing in time-series formats. However, efficiently and accurately extracting underlying patterns from massive, high-dimensional datasets to enable reliable predictions remains a core challenge for both academia and industry. Traditional time series models (e.g., ARIMA) struggle to capture complex nonlinear patterns, while deep learning models (e.g., LSTM, Transformer) face performance limitations due to hyperparameter selection and training strategies. Existing tuning processes lack automation and are prone to overfitting.This study proposes a systematic framework integrating automated hyperparameter optimization with deep learning training to enhance the accuracy, efficiency, and generalization capabilities of time series forecasting. Using data from Australia's electricity market (2006–2011), This article compared several models for load forecasting, including linear regression, Lasso, ARIMA, random forests, TCN, and an enhanced Transformer. This article used K-fold cross-validation, grid and Bayesian optimization, early stopping, and adaptive learning rate decay. The enhanced Transformer achieved the best performance (MAE ~70 MW, RMSE ~90 MW, R² ~0.995), outperforming other models. ARIMA underperformed due to its lack of exogenous variables. The Transformer showed robustness across seasons. Future work could combine TCN's local feature extraction with the Transformer's global dependency modeling.The time series forecasting framework proposed in this study effectively enhances prediction accuracy and model generalization capabilities. Transformer models demonstrate superior performance in forecasting complex dynamic changes. Future research may integrate the strengths of different models to expand their applications across multiple domains.
Downloads
References
[1] Liang Zhiyu, Wang Hongzhi. A Survey on Key Technologies for Time-Series Data Analysis in Intelligent IoT [J]. Journal of Computer Science and Technology, 2023, 12: 1-8.
[2] International Data Corporation. The Digitization of the World: From Edge to Core [M]. Seagate, 2018.
[3] International Data Corporation. IDC Vendor Spotlight [EB/OL]. Quantum, 2020-01.
[4] Wang Xinke, Mei Hongyan, Zhao Qin, et al. Research on Multivariate Long-Term Time Series Prediction Based on the TA-Informer Model [J/OL]. Computer Engineering and Applications,2025, 1-15. DOI: https://doi.org/10.1038/s41598-025-07654-7
[5] Fan Yaru, Xiong Zhihao. Data Prediction Methods Based on Mamba and Time Machine Models [J]. Journal of Southwest University for Nationalities (Natural Science Edition), 2025, 51(05):567-572.
[6] Cheng Tianle, Li Chengru, Fu Qianqian, etc Electric Vehicle Charging Load Prediction Based on CNN-LSTM-AM [J]. Automotive Electronics, 2025, (10): 62-64.
[7] Shi Yanli, Liu Xin, Zhao Jinxing Time series prediction algorithm based on cyclic step skipping network [J]. Computer Applications and Software, 2025, 42 (09): 324-330+368.
[8] Hyndman R J, Athanasopoulos G. Forecasting: Principles and Practice[M/OL]. Melbourne: OTexts, 2021. 3rd ed.
[9] James G, Witten D, Hastie T, Tibshirani R. An Introduction to Statistical Learning: With Applications in R[M].2nd ed. New York: Springer, 2021. DOI: https://doi.org/10.32614/CRAN.package.ISLR2
[10] Hong T, Fan S. Probabilistic electric load forecasting: A tutorial review[J]. International Journal of Forecasting, 2016, 32(3): 914–938. DOI: https://doi.org/10.1016/j.ijforecast.2015.11.011
[11] Pak A., Rad A. K., Nematollahi M. J., Mahmoudi M. Application of the Lasso regularisation technique in mitigating overfitting in air quality prediction models[J]. Scientific Reports, 2025, 15(1): 547. DOI: https://doi.org/10.1038/s41598-024-84342-y
[12] Ren P. Comparison and analysis of the accuracy of Lasso regression, Ridge regression and Elastic Net regression models in predicting students’ teaching quality achievement[J]. Applied and Computational Engineering, 2024, 51: 314–320. DOI: https://doi.org/10.54254/2755-2721/51/20241625
[13] Che J. X., Zhai H. C. WT-ARIMA combination modelling for short-term load forecasting[J]. IAENG International Journal of Computer Science, 2022, 49(2):8.
[14] Wang Q., Li S. Y., Li R. R., Jiang F. Underestimated impact of the COVID-19 on carbon emission reduction in developing countries – A novel assessment based on scenario analysis[J]. Environmental Research, 2022, 204(Pt A): 111990. DOI: https://doi.org/10.1016/j.envres.2021.111990
[15] Magalhães B., Bento P., Pombo J., Calado M. R., Mariano S. Short-Term Load Forecasting Based on Optimized Random Forest and Optimal Feature Selection[J]. Energies, 2024, 17(8): 1926. DOI: https://doi.org/10.3390/en17081926
[16] Dudek G. A comprehensive study of random forest for short-term load forecasting[J]. Energies, 2022, 15(20): 7547. DOI: https://doi.org/10.3390/en15207547
[17] Wang Y., Chen J., Chen X., Zeng X., Kong Y., Sun S., Guo Y., Liu Y. Short-term load forecasting for industrial customers based on TCN–LightGBM[J]. IEEE Transactions on Power Systems, 2021, 36(4): 1984–1997. DOI: https://doi.org/10.1109/TPWRS.2020.3028133
[18] Probst P., Wright M. N., Boulesteix A. L. Hyperparameters and tuning strategies for random forest[J]. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 2019, 9(3): e1301. DOI: https://doi.org/10.1002/widm.1301
[19] Smith H. L., Biggs P. J., French N. P., Smith A. N. H., Marshall J. C. Out of (the) bag—encoding categorical predictors impacts out-of-bag samples[J]. PeerJ Computer Science, 2024, 10: e2445. DOI: https://doi.org/10.7717/peerj-cs.2445
[20] Gao R., Du L., Suganthan P. N., Zhou Q., Yuen K. F. Random vector functional link neural network based ensemble deep learning for short-term load forecasting[J]. Expert Systems with Applications, 2022, 206: 117784. DOI: https://doi.org/10.1016/j.eswa.2022.117784
[21] Yan Q., Lu Z., Liu H., Guo H., Li W., Wang Z. An improved Feature-Time Transformer Encoder–Bi-LSTM for short-term forecasting of user-level integrated energy loads[J]. Energy and Buildings, 2023, 297:113396. DOI: https://doi.org/10.1016/j.enbuild.2023.113396
[22] Benidis K., Rangapuram S. S., Flunkert V., Wang Y., Maddix D. C., Gasthaus J., Januschowski T., et al. Deep learning for time series forecasting: Tutorial and literature survey[J]. ACM Computing Surveys, 2022, 55(1):1–38. DOI: https://doi.org/10.1145/3533382
[23] Zhang J., Pan X., Zhao J., Rho S., Hwang E. Transformer-based traffic forecasting model with simplified decoding structure for reduced complexity[J]. Frontiers in Neurorobotics, 2025, 19: 1527908. DOI: https://doi.org/10.3389/fnbot.2025.1527908
[24] Cao W., Liu H., Zhang X., Zeng Y., Ling X. Short-term residential load forecasting based on the fusion of load uncertainty feature extraction and meteorological factors[J]. Sustainability, 2025, 17(3): 1033. DOI: https://doi.org/10.3390/su17031033
[25] Dong Q., Huang R., Cui C., Towey D., Zhou L., Tian J., Wang J. Short-Term Electricity-Load Forecasting by Deep Learning: A Comprehensive Survey[J]. arXiv:2408.16202, 2024. DOI: https://doi.org/10.1016/j.engappai.2025.110980
[26] Hussein B. M., Shareef S. M. An Empirical Study on the Correlation between Early Stopping Patience and Epochs in Deep Learning[C]. ITM Web of Conferences, 2024, 64: 01003. DOI: https://doi.org/10.1051/itmconf/20246401003
[27] Zhou S. Y., Zhang Q. Y., Xiao P., Xu B. R., Luo G. S. UniLF: A novel short-term load forecasting model uniformly considering various features from multivariate load data[J]. Scientific Reports, 2025, 15: 4282. DOI: https://doi.org/10.1038/s41598-025-88566-4
[28] Park J. S., Bae H. J., Choi J. H., Kwon H. Y. Learning model combined with data clustering and dimensionality reduction for short-term electricity load forecasting[J]. Scientific Reports, 2025, 15: 3575. DOI: https://doi.org/10.1038/s41598-025-86982-0
[29] Australian Energy Market Operator (AEMO), Price and Demand Data Dictionary, Version 6.2, 2022. [Online]. Available: https://www.aemo.com.au.
[30] Baesmat K H, Shokoohi F, Farrokhi Z. SP-RF-ARIMA: A sparse random forest and ARIMA hybrid model for electric load forecasting[J]. Global Energy Interconnection, 2025, 8(3): 486-496. DOI: https://doi.org/10.1016/j.gloei.2025.04.003
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Highlights in Science, Engineering and Technology
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
