Predictive topology refinements in distributed stream processing system
DC Field | Value | Language |
---|---|---|
dc.contributor.author | Hanif, Muhammad | - |
dc.contributor.author | Lee, Choonhwa | - |
dc.contributor.author | Helal, Sumi | - |
dc.date.accessioned | 2022-07-07T11:15:46Z | - |
dc.date.available | 2022-07-07T11:15:46Z | - |
dc.date.created | 2021-05-12 | - |
dc.date.issued | 2020-11 | - |
dc.identifier.issn | 1932-6203 | - |
dc.identifier.uri | https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/144429 | - |
dc.description.abstract | Cloud computing has evolved the big data technologies to a consolidated paradigm with SPaaS (Streaming processing-as-a-service). With a number of enterprises offering cloud-based solutions to end-users and other small enterprises, there has been a boom in the volume of data, creating interest of both industry and academia in big data analytics, streaming applications, and social networking applications. With the companies shifting to cloud-based solutions as a service paradigm, the competition grows in the market. Good quality of service (QoS) is a must for the enterprises, as they strive to survive in a competitive environment. However, achieving reasonable QoS goals to meet SLA agreement cost-effectively is challenging due to variation in workload over time. This problem can be solved if the system has the ability to predict the workload for the near future. In this paper, we present a novel topology-refining scheme based on a workload prediction mechanism. Predictions are made through a model based on a combination of SVR, autoregressive, and moving average model with a feedback mechanism. Our streaming system is designed to increase the overall performance by making the topology refining robust to the incoming workload on the fly, while still being able to achieve QoS goals of SLA constraints. Apache Flink distributed processing engine is used as a testbed in the paper. The result shows that the prediction scheme works well for both workloads, i.e., synthetic as well as real traces of data. | - |
dc.language | 영어 | - |
dc.language.iso | en | - |
dc.publisher | PUBLIC LIBRARY SCIENCE | - |
dc.title | Predictive topology refinements in distributed stream processing system | - |
dc.type | Article | - |
dc.contributor.affiliatedAuthor | Lee, Choonhwa | - |
dc.identifier.doi | 10.1371/journal.pone.0240424 | - |
dc.identifier.scopusid | 2-s2.0-85095723356 | - |
dc.identifier.wosid | 000591376200046 | - |
dc.identifier.bibliographicCitation | PLOS ONE, v.15, no.11, pp.1 - 27 | - |
dc.relation.isPartOf | PLOS ONE | - |
dc.citation.title | PLOS ONE | - |
dc.citation.volume | 15 | - |
dc.citation.number | 11 | - |
dc.citation.startPage | 1 | - |
dc.citation.endPage | 27 | - |
dc.type.rims | ART | - |
dc.type.docType | Article | - |
dc.description.journalClass | 1 | - |
dc.description.isOpenAccess | Y | - |
dc.description.journalRegisteredClass | scie | - |
dc.description.journalRegisteredClass | scopus | - |
dc.relation.journalResearchArea | Science & Technology - Other Topics | - |
dc.relation.journalWebOfScienceCategory | Multidisciplinary Sciences | - |
dc.subject.keywordPlus | article | - |
dc.subject.keywordPlus | feedback system | - |
dc.subject.keywordPlus | human | - |
dc.subject.keywordPlus | prediction | - |
dc.subject.keywordPlus | workload | - |
dc.subject.keywordPlus | algorithm | - |
dc.subject.keywordPlus | cloud computing | - |
dc.subject.keywordPlus | computer network | - |
dc.subject.keywordPlus | quality control | - |
dc.identifier.url | https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0240424 | - |
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.
222, Wangsimni-ro, Seongdong-gu, Seoul, 04763, Korea+82-2-2220-1365
COPYRIGHT © 2021 HANYANG UNIVERSITY.
Certain data included herein are derived from the © Web of Science of Clarivate Analytics. All rights reserved.
You may not copy or re-distribute this material in whole or in part without the prior written consent of Clarivate Analytics.