VRKeyLogger: Virtual keystroke inference attack via eavesdropping controller usage pattern in WebVR

Abstract

WebVR is an emerging technology that allows users to experience VR (Virtual Reality) through typical web browsers, providing an integrated environment for various VR applications. One important problem of the VR technology is how to securely interact with users, in particular, implementing secure text input. A promising approach is to use a virtual keyboard rendered as a VR object. The VR user can enter certain text by clicking a sequence of virtual keys through the VR controllers, and the input text is handled in a secure way. However, despite the sensitivity of the input text, we found that there is a critical vulnerability that the VR controllers are not properly protected. The VR controller status can be disclosed to malicious entities, imposing a severe threat that an attacker's website can infer the input text by eavesdropping and analyzing the VR controller's movements. To accurately infer the input, the attacker should address two challenges: 1) determining which clicks correspond to the virtual keyboard and 2) identifying which key is pressed. In this paper, we propose a new keystroke inference attack framework, VRKeyLogger, that addresses such challenges with two key components: key-click classifier and key-click identifier. The key-click classifier effectively distinguishes clicks on the virtual keyboard based on the SVM classifier trained by the major features of the VR controller uses. The key-click identifier then accurately identifies which key is pressed by transforming the clicked position into the local coordinate system of the virtual keyboard. We implemented a proof-of-concept prototype and conducted a user study with nine participants. In the extensive user study with three real-world WebVR applications, our VRKeyLogger results in classification and identification accuracy of 93.98 and 96.8% on average, respectively. This implies that the proposed attack poses a serious threat to WebVR security. © 2023 Elsevier Ltd