Performance evaluation of the SSD-based swap system for big data processing

Abstract

Solid State Drives (SSDs) are quickly replacing HDDs not only in laptops but also in servers. Since SSD uses semiconductor, i.e., NAND flash memory, as its storage medium, it locates itself between memory and storage: it is faster but more expensive than HDD, and slower but cheaper than DRAM. Meanwhile, big data processing systems require both faster storage and larger memory. While lots of works have been conducted on evaluating the performance enhancement in big data processing systems by replacing HDD with SSD, the effect of SSD as a simple memory extension, i.e., Swap device, has not been studied, sufficiently. In this paper, we investigate the feasibility of using SSD as a swap device for big data processing systems through extensive experiments. Experimental results show that 1) incredibly large amount of IO's are issued to swap device when the physical memory cannot accommodate all data for processing, 2) SSD processes IO's to a swap device significantly faster than HDD, 3) using SSD as a storage device is a far better choice than using it as a swap device, and 4) enterprise SSD is not cost effective when it is used as a swap device.