Exploring OS-based full-system deterministic replay

Abstract

Modern computer systems have various sources of nondeterminism such as external inputs, concurrency in software and hardware, asynchronous interrupts and timing variations. With these sources of nondeterminism, many errors and bugs can remain undetected during development, manifesting in the form of corrupt data, hangs, crashes or other catastrophic results. Deterministic full-system replay helps in identifying the cause of such failures by reproducing a previously happened execution. Existing full-system deterministic replay schemes are based on either a special hardware implementation or a virtualization platform. Though beneficial, either they require non-trivial modifications to hardware or suffer from lack of reproducibility. This paper presents an innovative operating system (OS) based replay framework called Software Black Box (SBB) which is the first attempt to provide full-system replay without requiring any special hardware implementation or virtualization. It can reproduce the entire execution of a computer system including, user-level processes, OS functions and device drivers with instruction level accuracy. We implemented a prototype of SBB for Linux operating system ported to the ARM uniprocessor environment and evaluated its performance using Phoronix benchmark suites and some networking workloads. The results are promising, making it suitable for many purposes including debugging, testing, security and performance analysis.