Leveraging Prompt Engineering on Large Language Model for Semantic Log Parsing

Abstract

Log parsing serves as a pivotal initial stage in log analysis, identifying event templates from logs, which are unstructured (or semi-structured) texts containing crucial runtime data from software systems. Conventionally, research in this field has focused on discerning static description (i.e., template) and dynamic parameters (i.e., variables) within a log. Nonetheless, recent studies have confirmed that understanding the meaning of dynamic variables provides valuable information for log interpretation and analysis, which has consequently sparked the initiation of ‘semantic log parsing’. In this paper, we introduced LoGPT, a novel mechanism that perceives log parsing as a code generation task, subsequently identifying templates with semantic variables. LoGPT utilizes a few-shot prompt engineering approach with the Large Language Model (LLM) to facilitate semantic log parsing, thereby significantly reducing the need for labor-intensive manual labeling and resources. Our findings shed light on a fresh perspective in the domain of semantic log parsing.