Theoretical analysis of constructing wavelet synopsis on partitioned data sets

Abstract

Currently, the size of data becomes much larger and the distributed data processing is getting very important to manage the huge size of data. The MapReduce well known as Google's data processing environment is the most popular distributed platform with good scalability and fault tolerance. Many traditional algorithms in the single machine environment are being adopted to the MapReduce platform. In this paper we analyze a novel algorithm to generate wavelet synopses on the distributed MapReduce framework. Wavelet synopsis is one of the most popular dimensionality reduction methods and has been studied in various areas such as query optimization, approximate query answering, feature selection, etc. In the proposed algorithm, the wavelet synopsis can be calculated by a single MapReduce phase, and, by minimizing the amount of data communicated through the network of the distributed MapReduce platform, all computations are processed within almost linear time complexity. We theoretically study the properties of constructing wavelet synopsis on partitioned data sets and the correctness of the proposed algorithm.