CredsCache: Making OverlayFS scalable for containerized services

Abstract

OverlayFS, a Linux kernel driver for the Docker container, supports access control in Linux. User processes in the container must have appropriate privileges to access files in the backing file system. To that end, OverlayFS temporarily overrides credentials of a process with the mounter's credentials whenever the process accesses a file, inode, or directory. However, we found that this mechanism incurs severe overhead owing to contention in updating a shared reference counter for the mounter's credentials, which hinders OverlayFS scalability. In this paper, we propose a credential caching scheme, CredsCache, which greatly relieves contention by maintaining a per-process cached version of the mounter's credentials. In CredsCache, each process performs credential overriding and reverting using its own cached version without updating the shared reference counter. We implemented CredsCache in OverlayFS and evaluated its performance using micro- and macro-benchmarks. Using solid-state drive (SSD) backend storage, the micro-benchmark results showed that CredsCache achieved up to 23.3× and 54.7× higher performance for data and metadata I/O, respectively, than vanilla OverlayFS. Moreover, the macro-benchmark and real-world benchmark results showed that CredsCache produced up to 5.6× and 2.9× better performance for respective database and web server benchmarks than vanilla OverlayFS.