Development of a Visual Guidance System for Laparoscopic Surgical Palpation using Computer Vision

Document Type

Conference Proceeding

Publication Date



Currently, there are numerous obstacles to performing palpation during laparoscopic surgery. The laparoscopic interface does not allow access into a patient's body anything other than the tools that are inserted through small incisions. Palpation is a useful technique for augmenting surgical decision-making during laparoscopic surgery, especially when discerning operations involving cancerous tumors. In this study, a visual guidance system is proposed for use during laparoscopic palpation, specifically engineered to be part of a motion-based laparoscopic palpation technique. In particular, the YOLACT++ model is used to localize a target organ, the gall bladder, on a custom dataset of laparoscopic cholecystectomy. Our experiments showed an AP score of 90.10 for bounding boxes and 87.20 on masks. In terms of the speed performance, the model achieved a playback speed of approximately 20 fps, which translates to approximately 48 ms video latency. The palpation path guides are guidelines that are computer-generated within the identified organ, and show potential in helping the surgeon implement the palpation more accurately. Overall, this study demonstrates the potential of deep learning-based real-time image processing models to complete our motion-based laparoscopic palpation system, and to realize the promising role of artificial intelligence in surgical decision-making.