Abstract
Video-to-text and text-to-video retrieval are dominated by English benchmarks (e.g. DiDeMo, MSR-VTT) and recent multilingual corpora (e.g. RUDDER), yet Arabic remains underserved, lacking localized evaluation metrics. We introduce a three-stage framework, AutoArabic, utilizing state-of-the-art large language models (LLMs) to translate non-Arabic benchmarks into Modern Standard Arabic, reducing the manual revision required by nearly fourfold. The framework incorporates an error detection module that automatically flags potential translation errors with 97% accuracy. Applying the framework to DiDeMo, a video retrieval benchmark produces DiDeMo-AR, an Arabic variant with 40,144 fluent Arabic descriptions. An analysis of the translation errors is provided and organized into an insightful taxonomy to guide future Arabic localization efforts. We train a CLIP-style baseline with identical hyperparameters on the Arabic and English variants of the benchmark, finding a moderate performance gap (about 3 percentage points at Recall@1), indicating that Arabic localization preserves benchmark difficulty. We evaluate three post-editing budgets (zero/ flagged-only/ full) and find that performance improves monotonically with more post-editing, while the raw LLM output (zero-budget) remains usable. To ensure reproducibility to other languages, we made the code available at this https URL.
Abstract (translated)
视频到文本和文本到视频的检索任务目前主要依赖于英语基准（如DiDeMo、MSR-VTT）以及最近的多语言语料库（例如RUDDER），然而阿拉伯语却相对较少被关注，缺少本地化的评估指标。为此，我们提出了一种三阶段框架——AutoArabic，该框架利用最先进的大规模语言模型（LLMs）将非阿拉伯语基准翻译成现代标准阿拉伯语，从而减少了近四倍的手动修订工作量。该框架还包含一个错误检测模块，能够以97%的准确率自动标记潜在的翻译错误。 应用此框架对视频检索基准DiDeMo进行处理后，我们生成了名为DiDeMo-AR的阿拉伯语变体版本，其中包含了40,144个流利的阿拉伯文描述。我们还提供了一项关于翻译错误的分析，并将其组织成一个具有洞察力的分类法，以指导未来的阿拉伯语本地化工作。 我们在阿拉伯语和英语两种语言的基准上训练了一个CLIP风格的基础模型（使用相同的超参数），发现性能差距中等（Recall@1指标下约3个百分点），这表明阿拉伯语本地化保持了基准任务的难度不变。我们评估了三种后续编辑预算方案（零预算/仅标记错误/全面编辑）并发现，随着更多后续编辑工作的增加，性能会单调递增，而原始LLM输出（零预算版本）仍然可用。 为了确保该框架可以推广到其他语言的应用中，我们将代码开源至提供的网址。
URL
https://arxiv.org/abs/2509.16438
