High-performance IoT streaming data prediction system using Spark: a case study of air pollution
- Authors
- Jin, Ho-Yong; Jung, Eun-Sung; Lee, Duckki
- Issue Date
- Sep-2020
- Publisher
- SPRINGER LONDON LTD
- Keywords
- Long Short-Term Memory (LSTM); Distributed deep learning; Distributed Keras (Dist-Keras); Apache Spark
- Citation
- NEURAL COMPUTING & APPLICATIONS, v.32, no.17, pp.13147 - 13154
- Journal Title
- NEURAL COMPUTING & APPLICATIONS
- Volume
- 32
- Number
- 17
- Start Page
- 13147
- End Page
- 13154
- URI
- https://scholarworks.bwise.kr/hongik/handle/2020.sw.hongik/11575
- DOI
- 10.1007/s00521-019-04678-9
- ISSN
- 0941-0643
- Abstract
- Internet-of-Things (IoT) devices are becoming prevalent, and some of them, such as sensors, generate continuous time-series data, i.e., streaming data. These IoT streaming data are one of Big Data sources, and they require careful consideration for efficient data processing and analysis. Deep learning is emerging as a solution to IoT streaming data analytics. However, there is a persistent problem in deep learning that it takes a long time to learn neural networks. In this paper, we propose a high-performance IoT streaming data prediction system to improve the learning speed and to predict in real time. We showed the efficacy of the system through a case study of air pollution. The experimental results show that the modified LSTM autoencoder model shows the best performance compared to a generic LSTM model. We noticed that achieving the best performance requires optimizing many parameters, including learning rate, epoch, memory cell size, input timestep size, and the number of features/predictors. In that regard, we show that the high-performance data learning/prediction frameworks (e.g., Spark, Dist-Keras, and Hadoop) are essential to rapidly fine-tune a model for training and testing before real deployment of the model as data accumulate.
- Files in This Item
- There are no files associated with this item.
- Appears in
Collections - Graduate School > Software and Communications Engineering > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hongik/handle/2020.sw.hongik/11575)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.