Cache-aware stream consolidation in a server cluster for OTT VoD services

Abstract

The global proliferation of Over-the-Top (OTT) streaming services has provided unprecedented access to extensive high-definition video content across diverse devices and geographic regions, creating immense pressure on data center infrastructures. To handle the resulting surge in video traffic, OTT providers typically employ hierarchical architectures comprising origin data centers and geographically distributed Content Delivery Network (CDN) edge data centers, where both are structured as clusters of multiple servers with limited storage and processing capacity. To ensure high availability and adequate service capacity for popular videos in a server cluster, traditional approaches rely heavily on replication and load balancing strategies. However, these strategies often lead to significant inefficiencies in resource utilization due to excessive replica creation and degraded caching performance caused by stream fragmentation. In contrast, interval caching not only significantly improves caching performance by exploiting temporal locality between streams, but also reduces the need for replication by conserving resources. To fully exploit these benefits, we focus on stream consolidation as a means to create additional caching opportunities by reducing the temporal gaps between consecutive requests. To this end, we propose a novel cache-aware stream consolidation scheme that dynamically performs aggregation, replication, de-replication, and request steering operations based on real-time system load and the degree of stream consolidation for each video across servers. The main contributions of our proposed scheme are two-fold: First, it enhances resource utilization efficiency by applying an aggregation-first policy before resorting to replication, significantly reducing unnecessary replications while still maintaining adequate service capacity. Second, it improves caching efficiency by consolidating streams for the same video onto fewer servers, substantially shortening intervals between consecutive requests. Through extensive experiments, we demonstrate that our proposed scheme significantly outperforms conventional schemes including replication-only and replication with interval caching under varying parameters.