Automatic hybrid analysis technique to improve botnet code coverage using fake server

Abstract

The number of newly found malware and malware variants keeps increasing every year, and malware can be analyzed through static analysis or dynamic analysis. Malware developers use various packing techniques to avoid or to hinder static analysis and dynamic analysis needs to be used to analyze packed malware. In addition, malware that uses network communications to perform malicious actions that makes more difficult to be analyzed. Malware often hides malicious behaviors which are only triggered when certain conditions are satisfied, and this trigger-based malware usually communicates with the C2(Command and Control) server. With the increasing number of daily submitted malware, various malware analysis tools have also been developed steadily. In this paper, we propose a HyImCoCo (Hybrid Improve Code Coverage) which is a detection tool of trigger-based behaviors by satisfying the triggering conditions using hybrid analysis. IDApython is utilized to extract network-based features, and network-related instructions are modified to connect to our fake server called BKserver. The server can support TCP, UDP, IRC and HTTP protocols which are commonly used protocols by malware. HyImCoCo includes five modules which is Find Path module, Patch module, BKserver module, and CFG(Control Flow Graph) module. Experiments have shown that the proposed method can improve code coverage. With this tool, HyImCoCo helps to analyze the branch after recv by attracting malicious code to BKserver. We argue that our proposed tool, HyImCoCo can contribute to reduce the overheads of dynamic analysis through triggering malicious hidden behaviors in malware and increase to code coverage this helps analyze various paths.