An augmented reality visualization in neuronavigation system

Abstract

Image-guided surgery attempts to augment the human eyes with an enhanced vision of reality. It provides MR or CT axial views that lie beyond the exposed surfaces. The information is clinically quite helpful to a surgeon during operation. However, in the existing neuronavigation systems, the registration between the MR/CT imagery and the surgeon's real view should be made only in the surgeon's imagination and thus it may lead to confusion of spatial relationship. The proposed visualization in this paper is an attempt to resolve the problem of confusion. It provides a natural-looking vision of mixed reality where a pre-operative MR/CT image and computer-generated views of the surgical instruments are superimposed accurately onto a real intra-operative image. One of the key technologies in this system is how to accurately register various coordinate systems. In this paper, the coordinate systems of CCD cameras, MR/CT images, and surgical instruments are registered to a common framework by a 3D vision-based technique that allows us to freely change the position of involved objects during operation. The MR/CT images and the computer-generated surgical instruments are rendered in such a way that they look as if they are taken from the viewpoint of the CCD camera. Results: We put fiducial markers to obtain the geometric relationship between the MR/CT images and the calibrated CCD cameras (Saerim SR-101AN, NTSC quality). The coordinate of the surgical instrument is tracked on the basis of 3D computer vision. The accuracy of the registration is to a few pixels. It is shown that the proposed visualization system provides augmented views of an acceptable quality and thus is very useful for a variety of image-guided surgery. It would be interesting in the future to replace the pre-operative images with interventional MRI so that we can cope with any dynamic change of patient's body during the operation.