Abstract
Despite significant progress in optical character recognition (OCR) and computer vision systems, robustly recognizing text and identifying people in images taken in unconstrained \emph{in-the-wild} environments remain an ongoing challenge. However, such obstacles must be overcome in practical applications of vision systems, such as identifying racers in photos taken during off-road racing events. To this end, we introduce two new challenging real-world datasets - the off-road motorcycle Racer Number Dataset (RND) and the Muddy Racer re-iDentification Dataset (MUDD) - to highlight the shortcomings of current methods and drive advances in OCR and person re-identification (ReID) under extreme conditions. These two datasets feature over 6,300 images taken during off-road competitions which exhibit a variety of factors that undermine even modern vision systems, namely mud, complex poses, and motion blur. We establish benchmark performance on both datasets using state-of-the-art models. Off-the-shelf models transfer poorly, reaching only 15% end-to-end (E2E) F1 score on text spotting, and 33% rank-1 accuracy on ReID. Fine-tuning yields major improvements, bringing model performance to 53% F1 score for E2E text spotting and 79% rank-1 accuracy on ReID, but still falls short of good performance. Our analysis exposes open problems in real-world OCR and ReID that necessitate domain-targeted techniques. With these datasets and analysis of model limitations, we aim to foster innovations in handling real-world conditions like mud and complex poses to drive progress in robust computer vision. All data was sourced from this http URL, a website used by professional motorsports photographers, racers, and fans. The top-performing text spotting and ReID models are deployed on this platform to power real-time race photo search.
Abstract (translated)
尽管在光学字符识别（OCR）和计算机视觉系统方面取得了显著的进展，但在不受约束的野外环境中准确识别文本和识别人物仍然是一个持续的挑战。然而，在视觉系统的实际应用中， such 障碍必须被克服，例如在赛车照片中识别赛车手。为此，我们引入了两个新的具有挑战性的现实世界数据集——赛车手编号数据集（RND）和泥泞赛车手重新识别数据集（MUDD），以强调在极端条件下 OCR 和人物识别（ReID）方法的不足之处，推动在不受约束的环境中实现更好的识别性能。这两个数据集涵盖了在赛车比赛中拍摄的超过 6,300 张图像，这些图像呈现出各种因素，对即使是最先进的现代视觉系统也会产生影响，例如泥、复杂的姿势和运动模糊。我们在两个数据集上使用最先进的模型进行基准性能评估。通用的模型转移差，仅达到15%的端到端（E2E） F1 分数在文本检测中，而在 ReID 方面也只有33%的排名一准确率。微调带来重大改进，将模型的性能提高到53%的E2E文本检测和79%的排名一准确率在 ReID 上，但仍然存在不足。我们的分析揭示了在现实世界 OCR 和 ReID 中需要解决的问题，这需要领域特定的技术。有了这些数据集和模型限制的分析，我们旨在推动在处理类似泥和复杂姿态的实时情况方面的创新，以推动计算机视觉在实时情况下的进步。所有数据都来自这个链接，这是一个专业赛车摄影师、赛车手和粉丝使用的网站。在这个平台上，最优秀的文本检测和 ReID 模型部署用于实时赛车照片搜索。
URL
https://arxiv.org/abs/2402.08025
