Abstract
Diabetic foot ulcers (DFUs) are a leading cause of hospitalizations and lower limb amputations, placing a substantial burden on patients and healthcare systems. Early detection and accurate classification of DFUs are critical for preventing serious complications, yet many patients experience delays in receiving care due to limited access to specialized services. Telehealth has emerged as a promising solution, improving access to care and reducing the need for in-person visits. The integration of artificial intelligence and pattern recognition into telemedicine has further enhanced DFU management by enabling automatic detection, classification, and monitoring from images. Despite advancements in artificial intelligence-driven approaches for DFU image analysis, the application of large language models for DFU image transcription has not yet been explored. To address this gap, we introduce UlcerGPT, a novel multimodal approach leveraging large language and vision models for DFU image transcription. This framework combines advanced vision and language models, such as Large Language and Vision Assistant and Chat Generative Pre-trained Transformer, to transcribe DFU images by jointly detecting, classifying, and localizing regions of interest. Through detailed experiments on a public dataset, evaluated by expert clinicians, UlcerGPT demonstrates promising results in the accuracy and efficiency of DFU transcription, offering potential support for clinicians in delivering timely care via telemedicine.
Abstract (translated)
糖尿病足溃疡（DFUs）是医院化和下肢截肢的领先原因，对患者和医疗系统造成了沉重的负担。早期诊断和准确的分类DFUs对于预防严重并发症至关重要，然而许多患者由于获得专业服务受限而经历延迟接受治疗。远程医疗已成为一个有前景的解决方案，通过改善获得医疗服务的可访问性并减少需要亲自就诊，提高了医疗服务的可及性。将人工智能和模式识别融入远程医疗，进一步提高了DFU管理，通过使图像自动检测、分类和监测，从而实现这一目标。尽管在人工智能驱动的DFU图像分析方面取得了进步，但应用大型语言模型进行DFU图像转录的应用还尚不清楚。为了填补这一空白，我们引入了UlcerGPT，一种利用大型语言和视觉模型协同检测、分类和定位兴趣区域的全新多模态方法。这个框架结合了大型语言和视觉模型，如Large Language和 Vision Assistant和Chat Generative Pre-trained Transformer，通过共同检测、分类和定位感兴趣的区域对DFU图像进行转录。通过对一个公开数据集的详细实验，由专家临床医生进行评估，UlcerGPT在DFU转录的准确性和效率方面显示出良好的结果，为医生通过远程医疗及时交付护理提供了潜在支持。
URL
https://arxiv.org/abs/2410.01989
