An Efficient Topology Refining Scheme for Apache Flink
- Authors
- Hanif, Muhammad; Lee, Choonhwa
- Issue Date
- Nov-2018
- Publisher
- Institute of Electrical and Electronics Engineers Inc.
- Keywords
- Big Data; Cloud Computing; Distributed Computing; Stream Processing Engine; Topology
- Citation
- 9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018, pp.766 - 770
- Indexed
- SCOPUS
- Journal Title
- 9th International Conference on Information and Communication Technology Convergence: ICT Convergence Powered by Smart Intelligence, ICTC 2018
- Start Page
- 766
- End Page
- 770
- URI
- https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/149024
- DOI
- 10.1109/ICTC.2018.8539696
- ISSN
- 2162-1233
- Abstract
- In the past decade, there has been a boom in the volume of data and in the popularity of cloud applications with industry and academia keenly interested in big data analytics, streaming application, and social networking applications. This led to the emergence of real-time distributed stream processing systems such as Flink, Storm, Dataflow, and Samza. These systems process complex queries on streaming data sets to be distributed across multiple worker nodes in a cluster. Few of them provide adequate supports to adapt the topologies of stream processing tasks to changing input workload. We present an intelligent and efficient topology adjustment scheme which allow Flink framework to refine its topology on the basis of incoming workload. It is designed to increase the overall performance by making the refining of topology robust according to incoming workload streams on the fly, while maintaining SLA constraints. Apache Flink distributed processing engine is used as testbed in the paper. Our preliminary results indicate that the proposed system outperforms the existing default framework.
- Files in This Item
-
Go to Link
- Appears in
Collections - 서울 공과대학 > 서울 컴퓨터소프트웨어학부 > 1. Journal Articles
![qrcode](https://api.qrserver.com/v1/create-qr-code/?size=55x55&data=https://scholarworks.bwise.kr/hanyang/handle/2021.sw.hanyang/149024)
Items in ScholarWorks are protected by copyright, with all rights reserved, unless otherwise indicated.