SPST-CNN: Spatial pyramid based searching and tagging of liver’s intraoperative live views via CNN for minimal invasive surgery

Abstract

Laparoscopic liver surgery is challenging to perform because of compromised ability of the surgeon to localize subsurface anatomy due to minimal invasive visibility. While image guidance has the potential to address this barrier, intraoperative factors, such as insufflations and variable degrees of organ mobilization from supporting ligaments, may generate substantial deformation. The navigation ability in terms of searching and tagging within liver views has not been characterized, and current object detection methods do not account for the mechanics of how these features could be applied to the liver images. In this research, we have proposed spatial pyramid based searching and tagging of liver's intraoperative views using convolution neural network (SPST-CNN). By exploiting a hybrid combination of an image pyramid at input and spatial pyramid pooling layer at deeper stages of SPST-CNN, we reveal the gains of full-image representations for searching and tagging variable scaled liver live views. SPST-CNN provides pinpoint searching and tagging of intraoperative liver views to obtain up-to-date information about the location and shape of the area of interest. Downsampling input using image pyramid enables SPST-CNN framework to deploy input images with a diversity of resolutions for achieving scale-invariance feature. We have compared the proposed approach to the four recent state-of-the-art approaches and our method achieved better mAP up to 85.9%.