Abstract
Monitoring leftover products provides valuable insights that can be used to optimize future production. This is especially important for German bakeries because freshly baked goods have a very short shelf life. Automating this process can reduce labor costs, improve accuracy, and streamline operations. We propose automating this process using an object detection model to identify baked goods from images. However, the large diversity of German baked goods makes fully supervised training prohibitively expensive and limits scalability. Although open-vocabulary detectors (e.g., OWLv2, Grounding DINO) offer lexibility, we demonstrate that they are insufficient for our task. While motivated by bakeries, our work addresses the broader challenges of deploying computer vision in industries, where tasks are specialized and annotated datasets are scarce. We compile dataset splits with varying supervision levels, covering 19 classes of baked goods. We propose two training workflows to train an object detection model with limited supervision. First, we combine OWLv2 and Grounding DINO localization with image-level supervision to train the model in a weakly supervised manner. Second, we improve viewpoint robustness by fine-tuning on video frames annotated using Segment Anything 2 as a pseudo-label propagation model. Using these workflows, we train YOLOv11 for our detection task due to its favorable speed accuracy tradeoff. Relying solely on image-level supervision, the model achieves a mean Average Precision (mAP) of 0.91. Finetuning with pseudo-labels raises model performance by 19.3% under non-ideal deployment conditions. Combining these workflows trains a model that surpasses our fully-supervised baseline model under non-ideal deployment conditions, despite relying only on image-level supervision.
Abstract (translated)
监控剩余产品的状态可以提供有价值的见解，这些见解可用于优化未来的生产流程。对于德国的面包店来说，这一点尤为重要，因为新鲜烘焙的产品保质期非常短。自动化这一过程能够减少劳动力成本、提高准确性和简化操作流程。我们建议使用物体检测模型从图像中识别烘焙产品来实现这一目标。 然而，由于德国面包种类繁多，完全监督式的训练方法成本过高且不具备可扩展性。尽管开放词汇的检测器（如OWLv2和Grounding DINO）提供了灵活性，但我们证明它们对于我们的任务来说是不够的。虽然这项工作最初受到烘焙行业的启发，但它更广泛地解决了在行业中部署计算机视觉技术时面临的挑战：这些行业中的任务非常专业化且注释数据集稀缺。 为了应对这一问题，我们编制了不同监督水平的数据集拆分，涵盖了19类烘焙产品。为此，我们提出了两种有限监督训练物体检测模型的训练工作流程： 首先，我们将OWLv2和Grounding DINO定位与图像级监督相结合，以弱监督的方式训练模型。 其次，为了提高视角鲁棒性，我们在使用Segment Anything 2作为伪标签传播模型进行注释的视频帧上进行了微调。 利用这些工作流程，并考虑到YOLOv11在速度精度权衡方面的优势，我们针对检测任务对YOLOv11进行了训练。仅依赖图像级监督的情况下，该模型实现了0.91的平均精度（mAP）。通过使用伪标签进行微调，在非理想部署条件下使模型性能提高了19.3%。 结合这些工作流程，即使在只依靠图像级监督的情况下，也能训练出一个超过我们完全监督基准模型性能的模型。这种方法特别适用于解决那些标注数据稀缺、任务具有特殊性的行业的挑战。
URL
https://arxiv.org/abs/2602.09979
