Kafka-Thor: A Kafka-based In-Edge Data Streaming Platform for Enhanced V2X Services

Abstract

Enhanced Vehicle-to-Everything (V2X) services require ultra-low latency and robust reliability in end-to-end (E2E) data delivery from producers (e.g., vehicles and roadside units) to consumers (e.g., in-car V2X service applications). One of the candidate data delivery architectures that can meet the stringent requirements is restricting the data delivery path within the network edge by letting an edge server relay data between sources and destinations in its coverage. To that end, edge servers need to equip two important functionalities: 1) a reliable and lightweight data transport protocol to collect data from numerous data sources, and 2) an efficient data streaming platform that delivers collected data to consumers with high throughput and low latency. In this work, we design and implement Kafka- Thor, a Kafka-based high-performance data streaming platform, specifically designed for edge servers to collect and deliver timecritical data for enhanced V2X services. The platform reduces E2E data delivery latency by optimizing the data delivery architecture in Kafka using two novel technologies, the singlepoller, multi-worker (SPMW) architecture and service-specific dynamic batching (SS-batching). Experimental results show that Kafka-Thor significantly improves latency and reliability in E2E data delivery, making enhanced V2X services feasible.