Abstract
This paper presents a novel multi modal deep learning framework for enhanced agricultural pest detection, combining tiny-BERT's natural language processing with R-CNN and ResNet-18's image processing. Addressing limitations of traditional CNN-based visual methods, this approach integrates textual context for more accurate pest identification. The R-CNN and ResNet-18 integration tackles deep CNN issues like vanishing gradients, while tiny-BERT ensures computational efficiency. Employing ensemble learning with linear regression and random forest models, the framework demonstrates superior discriminate ability, as shown in ROC and AUC analyses. This multi modal approach, blending text and image data, significantly boosts pest detection in agriculture. The study highlights the potential of multi modal deep learning in complex real-world scenarios, suggesting future expansions in diversity of datasets, advanced data augmentation, and cross-modal attention mechanisms to enhance model performance.
Abstract (translated)
本文提出了一种新颖的多模态深度学习框架,用于增强农业害虫检测,将 tiny-BERT 的自然语言处理与 R-CNN 和 ResNet-18 的图像处理相结合。该方法解决了传统 CNN 视觉方法的局限性,并引入了文本上下文以实现更精确的害虫识别。R-CNN 和 ResNet-18 的集成解决了深度 CNN 问题,如消失的梯度,而 tiny-BERT 保证了计算效率。通过线性回归和随机森林模型的集成,该框架展示了卓越的判别能力,如图论和 AUC 分析结果所示。这种多模态方法结合了文本和图像数据,显著提高了农业中的害虫检测。本研究突出了多模态深度学习在复杂现实场景中的潜力,建议在未来增加数据集的多样性、高级数据增强和跨模态关注机制,以提高模型性能。
URL
https://arxiv.org/abs/2312.10948
