03/27/2025
By Xiaokun Chen

The Kennedy College of Sciences, Department of Miner School of Computer & Information Sciences, invites you to attend a Master’s thesis defense by Xiaokun Chen on “Development and Validation of a Large Language and Vision Model for Colonoscopy Analytics."

Candidate Name: Xiaokun Chen
Degree: Master’s
Defense Date: Wednesday, April 9, 2025
Time: 9-11 a.m.
Location: Zoom link
Meeting ID: 572 450 1700
Passcode: xiaokun

Thesis/Dissertation Title: Development and Validation of a Large Language and Vision Model for Colonoscopy Analytics

Committee:

Advisor: Dr. Yu Cao, Miner School of Computer & Information Sciences, University of Massachusetts - Lowell

Committee Members:
Prof. Benyuan Liu, Ph.D., Miner School of Computer & Information Sciences, University of Massachusetts - Lowell
Prof. Ming Shao, Ph.D., Miner School of Computer & Information Sciences, University of Massachusetts - Lowell

Abstract:
Colorectal cancer is among the leading causes of cancer-related deaths worldwide, and its incidence continues to rise, especially in developing countries. Colonoscopy serves as a critical tool for both prevention and early treatment, but limited availability of trained medical professionals significantly restricts access. Recent advancements in artificial intelligence (AI), particularly multimodal language models (MLMs), offer promising solutions to these limitations. ColonGPT, initially developed in the research study Frontiers in Intelligent Colonoscopy by Ge-Peng Ji et al., is an innovative multimodal language model specifically designed for colonoscopy applications. ColonGPT integrates visual classification, object detection, and image captioning capabilities into a unified conversational interface, thereby streamlining interactions for medical professionals and enhancing patient engagement. This thesis aims to significantly enhance ColonGPT’s capabilities by expanding its training dataset with supplementary data, optimizing training strategies, and employing advanced AI frameworks, including SigLIP vision encoders and state-of-the-art visual instruction-tuning techniques. We further investigate parameter-efficient fine-tuning strategies, such as Low-Rank Adaptation (LoRA), to improve model efficiency and performance. The outcomes of this research will establish a new benchmark for AI-enhanced colonoscopy, laying the groundwork for sophisticated applications such as automated medical report generation and diagnostic assistance, especially benefiting regions with limited medical resources.