NMT
-
Asymptotically Smaller Encodings for Graph Problems and Scheduling
Abstract
We show how several graph problems (e.g., vertex-cover, independent-set, $k$-coloring) can be encoded into CNF using only $O(|V|^2 / \lg |V|)$ many clauses, as opposed to the $\Omega(|V|^2)$ constraints used by standard encodings. This somewhat surprising result is a simple consequence of a result of ErdÅs, Chung, and Spencer (1983) about biclique coverings of graphs, and opens theoretical avenues to understand the success of "Bounded Variable Addition'' (Manthey, Heule, and Biere, 2012) as a preprocessing tool. Finally, we show a novel encoding for independent sets in some dense interval graphs using only $O(|V| \lg |V|)$ clauses (the direct encoding uses $\Omega(|V|^2)$), which we have successfully applied to a string-compression encoding posed by Bannai et al. (2022). As a direct byproduct, we obtain a reduction in the encoding size of a scheduling problem posed by Mayank and Modal (2020) from $O(NMT^2)$ to $O(NMT + M T^2 \lg T)$, where $N$ is the number of tasks, $T$ the total timespan, and $M$ the number of machines.
Abstract (translated)
我们展示了如何将若干图论问题（例如顶点覆盖、独立集、$k$-着色）编码为CNF（合取范式），仅使用$O(|V|^2 / \lg |V|)$数量的子句，相比之下传统的编码方法需要$\Omega(|V|^2)$的数量约束。这一令人惊讶的结果是图论中ErdÅ¡os、Chung和Spencer（1983年）关于双完全子图覆盖结果的一个简单推论，并为我们理解“有限变量添加”（Manthey, Heule 和 Biere，2012年）作为预处理工具的成功提供了一个理论途径。最后，我们提出了一种新颖的编码方法，在某些密集区间图中独立集仅使用$O(|V| \lg |V|)$数量的子句（直接编码则需要$\Omega(|V|^2)$），并且这种编码已被成功应用于Bannai等人（2022年）提出的字符串压缩问题。作为直接成果，我们还得到了Mayank和Modal（2020年）提出的一个调度问题的编码大小从$O(NMT^2)$减少到$O(NMT + M T^2 \lg T)$的结果，其中$N$表示任务数量,$T$是总时间跨度，而$M$则是机器的数量。
URL
https://arxiv.org/abs/2506.14042
PDF
-
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT
2025-06-05 12:02:01Miko{\l}aj Pokrywka, Wojciech Kusa, Mieszko Rutkowski, Miko{\l}aj KoszowskiAbstract
Neural Machine Translation (NMT) has improved translation by using Transformer-based models, but it still struggles with word ambiguity and context. This problem is especially important in domain-specific applications, which often have problems with unclear sentences or poor data quality. Our research explores how adding information to models can improve translations in the context of e-commerce data. To this end we create ConECT -- a new Czech-to-Polish e-commerce product translation dataset coupled with images and product metadata consisting of 11,400 sentence pairs. We then investigate and compare different methods that are applicable to context-aware translation. We test a vision-language model (VLM), finding that visual context aids translation quality. Additionally, we explore the incorporation of contextual information into text-to-text models, such as the product's category path or image descriptions. The results of our study demonstrate that the incorporation of contextual information leads to an improvement in the quality of machine translation. We make the new dataset publicly available.
Abstract (translated)
神经机器翻译（NMT）通过使用基于Transformer的模型在翻译方面取得了进步，但仍面临词汇歧义和上下文理解的问题。这一问题在特定领域的应用中尤为关键，这些领域常常遇到表述不清或数据质量差的情况。我们的研究探讨了如何通过向模型添加信息来提升电子商务数据中的翻译质量。为此，我们创建了ConECT——一个新的从捷克语到波兰语的电子商务产品翻译数据集，并与图像和产品元数据相结合，包含11,400对句子。接着，我们调查并比较了不同的适用于上下文感知翻译的方法。 我们在测试中使用了一种视觉-语言模型（VLM），发现视觉信息有助于提升翻译质量。此外，我们也探索了将上下文信息整合进文本到文本模型中的方法，比如产品的类别路径或图像描述的集成。研究结果表明，通过引入上下文信息能够显著提高机器翻译的质量。 我们计划公开发布这一新数据集。
URL
https://arxiv.org/abs/2506.04929
PDF
-
A Representation Level Analysis of NMT Model Robustness to Grammatical Errors
2025-05-27 14:10:30Abderrahmane Issam, Yusuf Can Semerci, Jan Scholtes, Gerasimos SpanakisAbstract
Understanding robustness is essential for building reliable NLP systems. Unfortunately, in the context of machine translation, previous work mainly focused on documenting robustness failures or improving robustness. In contrast, we study robustness from a model representation perspective by looking at internal model representations of ungrammatical inputs and how they evolve through model layers. For this purpose, we perform Grammatical Error Detection (GED) probing and representational similarity analysis. Our findings indicate that the encoder first detects the grammatical error, then corrects it by moving its representation toward the correct form. To understand what contributes to this process, we turn to the attention mechanism where we identify what we term Robustness Heads. We find that Robustness Heads attend to interpretable linguistic units when responding to grammatical errors, and that when we fine-tune models for robustness, they tend to rely more on Robustness Heads for updating the ungrammatical word representation.
Abstract (translated)
理解稳健性对于构建可靠的自然语言处理系统至关重要。不幸的是，在机器翻译的背景下，之前的工作主要集中在记录稳健性的失败案例或改进稳健性上。相比之下，我们从模型表示的角度研究稳健性，通过观察非语法输入在各个模型层中的内部表示以及它们如何演变来实现这一目标。为此，我们进行了语法错误检测（GED）探测和表征相似度分析。我们的发现表明，编码器首先检测到语法错误，然后通过将表示形式移动到正确的形式来进行修正。为了了解推动这一过程的因素，我们转向注意力机制，在那里我们识别出了我们称之为“稳健性头”的部分。我们发现，“稳健性头”在响应语法错误时会关注可解释的语言单元，并且当我们针对稳健性进行模型微调时，它们倾向于更多地依赖于“稳健性头”来更新非语法单词的表示形式。
URL
https://arxiv.org/abs/2505.21224
PDF
-
Low-Resource NMT: A Case Study on the Written and Spoken Languages in Hong Kong
Abstract
The majority of inhabitants in Hong Kong are able to read and write in standard Chinese but use Cantonese as the primary spoken language in daily life. Spoken Cantonese can be transcribed into Chinese characters, which constitute the so-called written Cantonese. Written Cantonese exhibits significant lexical and grammatical differences from standard written Chinese. The rise of written Cantonese is increasingly evident in the cyber world. The growing interaction between Mandarin speakers and Cantonese speakers is leading to a clear demand for automatic translation between Chinese and Cantonese. This paper describes a transformer-based neural machine translation (NMT) system for written-Chinese-to-written-Cantonese translation. Given that parallel text data of Chinese and Cantonese are extremely scarce, a major focus of this study is on the effort of preparing good amount of training data for NMT. In addition to collecting 28K parallel sentences from previous linguistic studies and scattered internet resources, we devise an effective approach to obtaining 72K parallel sentences by automatically extracting pairs of semantically similar sentences from parallel articles on Chinese Wikipedia and Cantonese Wikipedia. We show that leveraging highly similar sentence pairs mined from Wikipedia improves translation performance in all test sets. Our system outperforms Baidu Fanyi's Chinese-to-Cantonese translation on 6 out of 8 test sets in BLEU scores. Translation examples reveal that our system is able to capture important linguistic transformations between standard Chinese and spoken Cantonese.
Abstract (translated)
香港的大多数居民能够阅读和书写标准汉语，但在日常生活中主要使用粤语作为口语。粤语可以通过汉字转写成所谓的“书面粤语”。书面粤语在词汇和语法上与标准书面中文有显著差异。随着网络世界的不断发展，书面粤语的应用越来越明显。普通话使用者和粤语使用者之间的互动日益增加，这导致了对中文与粤语文本自动翻译的明确需求。本文描述了一种基于Transformer的神经机器翻译（NMT）系统，用于将书面中文翻译成书面粤文。 鉴于汉语文本和粤语文本的平行数据极为稀少，此研究的主要焦点在于为NMT准备大量训练数据的努力上。除了从先前的语言学研究和散落于互联网上的资源中收集28,000个平行句子外，我们还设计了一种有效的方法，通过自动提取来自中文维基百科和粤语维基百科的并行文章中的语义相似句子对来获得72,000个平行句子。结果显示，利用从维基百科中挖掘出的高度相似句对能提高所有测试集上的翻译性能。 我们的系统在BLEU评分上，在8组测试集中有6组超越了百度翻译的中文到粤文翻译效果。翻译示例表明，该系统能够捕捉到标准汉语与口语粤语之间的重要的语言转换。
URL
https://arxiv.org/abs/2505.17816
PDF
-
In-Domain African Languages Translation Using LLMs and Multi-armed Bandits
2025-05-21 03:33:27Pratik Rakesh Singh, Kritarth Prasad, Mohammadi Zaki, Pankaj WasnikAbstract
Neural Machine Translation (NMT) systems face significant challenges when working with low-resource languages, particularly in domain adaptation tasks. These difficulties arise due to limited training data and suboptimal model generalization, As a result, selecting an optimal model for translation is crucial for achieving strong performance on in-domain data, particularly in scenarios where fine-tuning is not feasible or practical. In this paper, we investigate strategies for selecting the most suitable NMT model for a given domain using bandit-based algorithms, including Upper Confidence Bound, Linear UCB, Neural Linear Bandit, and Thompson Sampling. Our method effectively addresses the resource constraints by facilitating optimal model selection with high confidence. We evaluate the approach across three African languages and domains, demonstrating its robustness and effectiveness in both scenarios where target data is available and where it is absent.
Abstract (translated)
神经机器翻译（NMT）系统在处理低资源语言时，特别是在领域适应任务中面临重大挑战。这些困难主要是由于训练数据有限以及模型泛化能力不足所导致的。因此，在没有可行或实际微调的情况下，为特定领域选择最佳翻译模型对于实现强大性能至关重要。 在这篇论文中，我们探讨了使用基于多臂赌博机（bandit-based）算法的选择策略来挑选最适合给定领域的NMT模型，这些算法包括上置信界（Upper Confidence Bound）、线性上置信界（Linear UCB）、神经线性赌博机（Neural Linear Bandit）以及汤普森采样（Thompson Sampling）。我们的方法通过在资源有限的情况下有效进行最佳模型选择，从而高效地解决了这一问题。 我们对三种非洲语言和领域进行了评估，并展示了该方法的鲁棒性和有效性，无论是有目标数据还是无目标数据的情况都能取得良好效果。
URL
https://arxiv.org/abs/2505.15069
PDF
-
THOR-MoE: Hierarchical Task-Guided and Context-Responsive Routing for Neural Machine Translation
2025-05-20 10:27:19Yunlong Liang, Fandong Meng, Jie ZhouAbstract
The sparse Mixture-of-Experts (MoE) has achieved significant progress for neural machine translation (NMT). However, there exist two limitations in current MoE solutions which may lead to sub-optimal performance: 1) they directly use the task knowledge of NMT into MoE (\emph{e.g.}, domain/linguistics-specific knowledge), which are generally unavailable at practical application and neglect the naturally grouped domain/linguistic properties; 2) the expert selection only depends on the localized token representation without considering the context, which fully grasps the state of each token in a global view. To address the above limitations, we propose THOR-MoE via arming the MoE with hierarchical task-guided and context-responsive routing policies. Specifically, it 1) firstly predicts the domain/language label and then extracts mixed domain/language representation to allocate task-level experts in a hierarchical manner; 2) injects the context information to enhance the token routing from the pre-selected task-level experts set, which can help each token to be accurately routed to more specialized and suitable experts. Extensive experiments on multi-domain translation and multilingual translation benchmarks with different architectures consistently demonstrate the superior performance of THOR-MoE. Additionally, the THOR-MoE operates as a plug-and-play module compatible with existing Top-$k$~\cite{shazeer2017} and Top-$p$~\cite{huang-etal-2024-harder} routing schemes, ensuring broad applicability across diverse MoE architectures. For instance, compared with vanilla Top-$p$~\cite{huang-etal-2024-harder} routing, the context-aware manner can achieve an average improvement of 0.75 BLEU with less than 22\% activated parameters on multi-domain translation tasks.
Abstract (translated)
稀疏的专家混合（Mixture-of-Experts，MoE）在神经机器翻译（NMT）方面取得了显著进展。然而，当前的MoE解决方案存在两个可能导致性能次优的问题：1) 它们直接将NMT的任务知识（例如，特定领域的或语言学的知识）用于MoE中，这些信息在实际应用中通常不可用，并且忽视了自然分组的领域/语言特性；2) 专家选择仅依赖于局部标记表示，而不考虑上下文，这无法从全局视角全面掌握每个标记的状态。为了解决上述问题，我们提出了通过配备层次化任务引导和上下文响应路由策略来增强MoE能力的THOR-MoE模型。 具体来说，1）该方法首先预测领域/语言标签，然后提取混合领域的表示以分层方式分配任务级别的专家；2）注入上下文信息以增强预选的任务级专家集合中的标记路由，这可以帮助每个标记更准确地路由到更加专业化且合适的专家。在不同架构上的多域翻译和跨语言翻译基准测试中进行了广泛的实验，结果一致表明THOR-MoE具有卓越的性能。 此外，THOR-MoE作为即插即用模块与现有的Top-$k$~\cite{shazeer2017} 和 Top-$p$~\cite{huang-etal-2024-harder} 路由方案兼容，在广泛的MoE架构中确保了广泛应用性。例如，与原始的Top-$p$~\cite{huang-etal-2024-harder} 路由相比，上下文感知方式在多域翻译任务上平均提高了0.75个BLEU分数，并且激活参数少于22%。 这段文字详细介绍了THOR-MoE模型如何改进现有的MoE架构以适应神经机器翻译的需求。通过引入层次化的任务引导和增强的上下文响应机制，该模型不仅能够更好地利用语言学和领域知识，还能更精确地为每个标记选择合适的专家，从而提高翻译质量。实验结果表明了THOR-MoE在多域与跨语言翻译中的优越性能，并且其灵活的设计使其适用于多种路由方案，增加了实际应用中的通用性和效率。
URL
https://arxiv.org/abs/2505.14173
PDF
-
Combining the Best of Both Worlds: A Method for Hybrid NMT and LLM Translation
2025-05-19 06:50:52Zhanglin Wu, Daimeng Wei, Xiaoyu Chen, Hengchao Shang, Jiaxin Guo, Zongyao Li, Yuanchang Luo, Jinlong Yang, Zhiqiang Rao, Hao YangAbstract
Large language model (LLM) shows promising performances in a variety of downstream tasks, such as machine translation (MT). However, using LLMs for translation suffers from high computational costs and significant latency. Based on our evaluation, in most cases, translations using LLMs are comparable to that generated by neural machine translation (NMT) systems. Only in particular scenarios, LLM and NMT models show respective advantages. As a result, integrating NMT and LLM for translation and using LLM only when necessary seems to be a sound solution. A scheduling policy that optimizes translation result while ensuring fast speed and as little LLM usage as possible is thereby required. We compare several scheduling policies and propose a novel and straightforward decider that leverages source sentence features. We conduct extensive experiments on multilingual test sets and the result shows that we can achieve optimal translation performance with minimal LLM usage, demonstrating effectiveness of our decider.
Abstract (translated)
大型语言模型（LLM）在多种下游任务中表现出色，例如机器翻译（MT）。然而，使用LLM进行翻译面临着高昂的计算成本和显著的延迟问题。根据我们的评估，在大多数情况下，基于LLM的翻译效果与神经机器翻译（NMT）系统生成的效果相当。只有在特定场景下，LLM和NMT模型各自展现出优势。 因此，将NMT和LLM结合起来用于翻译，并仅在必要时使用LLM似乎是一个合理的解决方案。为了优化翻译结果并确保快速的速度以及尽可能少地使用LLM，需要制定一个调度策略。我们对比了几个不同的调度策略，并提出了一种新的、简单的决策机制，该机制利用源语言句子的特征来决定是否调用LLM。 我们在多语种测试集上进行了广泛的实验，结果显示我们可以实现最优翻译性能的同时减少对LLM的使用，证明了我们的决策机制的有效性。
URL
https://arxiv.org/abs/2505.13554
PDF
-
Are We Paying Attention to Her? Investigating Gender Disambiguation and Attention in Machine Translation
2025-05-13 13:17:23Chiara Manna, Afra Alishahi, Fr\'ed\'eric Blain, Eva VanmassenhoveAbstract
While gender bias in modern Neural Machine Translation (NMT) systems has received much attention, traditional evaluation metrics do not to fully capture the extent to which these systems integrate contextual gender cues. We propose a novel evaluation metric called Minimal Pair Accuracy (MPA), which measures the reliance of models on gender cues for gender disambiguation. MPA is designed to go beyond surface-level gender accuracy metrics by focusing on whether models adapt to gender cues in minimal pairs -- sentence pairs that differ solely in the gendered pronoun, namely the explicit indicator of the target's entity gender in the source language (EN). We evaluate a number of NMT models on the English-Italian (EN--IT) language pair using this metric, we show that they ignore available gender cues in most cases in favor of (statistical) stereotypical gender interpretation. We further show that in anti-stereotypical cases, these models tend to more consistently take masculine gender cues into account while ignoring the feminine cues. Furthermore, we analyze the attention head weights in the encoder component and show that while all models encode gender information to some extent, masculine cues elicit a more diffused response compared to the more concentrated and specialized responses to feminine gender cues.
Abstract (translated)
尽管现代神经机器翻译（NMT）系统中的性别偏见已受到广泛关注，但传统的评估指标并不能完全捕捉这些系统在整合上下文性别线索方面的程度。为此，我们提出了一种新的评估指标——最小成对准确度（MPA），该指标衡量模型依赖于性别线索进行性别消歧的程度。MPA旨在超越表面的性别准确性指标，通过关注模型如何根据最小对中的性别代词调整其行为来实现这一点——即源语言中目标实体性别的明确指示器。 我们使用这一新指标评估了多种NMT模型在英语-意大利语（EN--IT）语言对上的性能。结果表明，在大多数情况下，这些模型倾向于忽略可用的性别线索而依赖于统计意义上的刻板印象进行性别解读。此外，我们在反向刻板印象的情况下还发现，这些模型更一致地考虑男性性别线索并忽视女性性别线索。 进一步分析编码器组件中的注意力头权重后我们发现，虽然所有模型都在某种程度上编码了性别信息，但男性提示会引发更为分散的响应，而女性性别的提示则引发了更加集中和专业的反应。
URL
https://arxiv.org/abs/2505.08546
PDF
-
Do Not Change Me: On Transferring Entities Without Modification in Neural Machine Translation -- a Multilingual Perspective
2025-05-09 12:47:13Dawid Wisniewski, Mikolaj Pokrywka, Zofia RostekAbstract
Current machine translation models provide us with high-quality outputs in most scenarios. However, they still face some specific problems, such as detecting which entities should not be changed during translation. In this paper, we explore the abilities of popular NMT models, including models from the OPUS project, Google Translate, MADLAD, and EuroLLM, to preserve entities such as URL addresses, IBAN numbers, or emails when producing translations between four languages: English, German, Polish, and Ukrainian. We investigate the quality of popular NMT models in terms of accuracy, discuss errors made by the models, and examine the reasons for errors. Our analysis highlights specific categories, such as emojis, that pose significant challenges for many models considered. In addition to the analysis, we propose a new multilingual synthetic dataset of 36,000 sentences that can help assess the quality of entity transfer across nine categories and four aforementioned languages.
Abstract (translated)
当前的机器翻译模型在大多数场景中提供了高质量的输出。然而，它们仍然面临一些特定问题，例如检测哪些实体在翻译过程中不应被更改。在这篇论文中，我们研究了流行神经机器翻译（NMT）模型的能力，包括来自OPUS项目、Google Translate、MADLAD和EuroLLM的模型，在四种语言之间进行翻译时保留实体（如网址地址、IBAN号码或电子邮件）的效果。这四种语言分别是英语、德语、波兰语和乌克兰语。我们分析了这些流行NMT模型在准确性方面的质量，讨论了它们所犯的错误，并探讨了导致错误的原因。我们的分析突出了诸如表情符号等特定类别，这对许多考虑中的模型构成了重大挑战。除了这项分析外，我们还提出了一套新的多语言合成数据集，包含36,000个句子，可以用来评估九种类型实体在上述四种语言之间的传输质量。
URL
https://arxiv.org/abs/2505.06010
PDF
-
TopicVD: A Topic-Based Dataset of Video-Guided Multimodal Machine Translation for Documentaries
2025-05-09 01:31:02Jinze Lv, Jian Chen, Zi Long, Xianghua Fu, Yin ChenAbstract
Most existing multimodal machine translation (MMT) datasets are predominantly composed of static images or short video clips, lacking extensive video data across diverse domains and topics. As a result, they fail to meet the demands of real-world MMT tasks, such as documentary translation. In this study, we developed TopicVD, a topic-based dataset for video-supported multimodal machine translation of documentaries, aiming to advance research in this field. We collected video-subtitle pairs from documentaries and categorized them into eight topics, such as economy and nature, to facilitate research on domain adaptation in video-guided MMT. Additionally, we preserved their contextual information to support research on leveraging the global context of documentaries in video-guided MMT. To better capture the shared semantics between text and video, we propose an MMT model based on a cross-modal bidirectional attention module. Extensive experiments on the TopicVD dataset demonstrate that visual information consistently improves the performance of the NMT model in documentary translation. However, the MMT model's performance significantly declines in out-of-domain scenarios, highlighting the need for effective domain adaptation methods. Additionally, experiments demonstrate that global context can effectively improve translation performance. % Dataset and our implementations are available at this https URL
Abstract (translated)
现有的多模态机器翻译（MMT）数据集主要由静态图像或短视频片段组成，缺乏跨不同领域和主题的大量视频数据。因此，这些数据集无法满足诸如纪录片翻译等实际场景中MMT任务的需求。为此，在这项研究中我们开发了基于话题的视频支持多模态机器翻译的数据集TopicVD，旨在推动该领域的研究进展。我们在文档片中收集并整理了视频-字幕配对，并将它们归类为八个话题（例如经济和自然），以便于研究视频引导下的MMT领域适应性问题。同时我们保留了它们的上下文信息以支持在视频指导下利用纪录片全局背景的研究。 为了更好地捕捉文本与视频之间的共同语义，我们提出了一种基于跨模态双向注意力模块的MMT模型。在TopicVD数据集上的广泛实验表明，在纪录片翻译中视觉信息能够持续改善NMT（神经机器翻译）模型的表现力。然而，当遇到域外场景时，MMT模型表现显著下降，这凸显了有效领域适应方法的需求。此外，实验还显示全局背景能有效地提升翻译性能。 这项研究的数据集及相关实现可以在这个链接下获取：[请在此插入数据集和代码的实际访问网址]
URL
https://arxiv.org/abs/2505.05714
PDF
-
Data Augmentation With Back translation for Low Resource languages: A case of English and Luganda
2025-05-05 08:47:52Richard Kimera, Dongnyeong Heo, Daniela N. Rim, Heeyoul ChoiAbstract
In this paper,we explore the application of Back translation (BT) as a semi-supervised technique to enhance Neural Machine Translation(NMT) models for the English-Luganda language pair, specifically addressing the challenges faced by low-resource languages. The purpose of our study is to demonstrate how BT can mitigate the scarcity of bilingual data by generating synthetic data from monolingual corpora. Our methodology involves developing custom NMT models using both publicly available and web-crawled data, and applying Iterative and Incremental Back translation techniques. We strategically select datasets for incremental back translation across multiple small datasets, which is a novel element of our approach. The results of our study show significant improvements, with translation performance for the English-Luganda pair exceeding previous benchmarks by more than 10 BLEU score units across all translation directions. Additionally, our evaluation incorporates comprehensive assessment metrics such as SacreBLEU, ChrF2, and TER, providing a nuanced understanding of translation quality. The conclusion drawn from our research confirms the efficacy of BT when strategically curated datasets are utilized, establishing new performance benchmarks and demonstrating the potential of BT in enhancing NMT models for low-resource languages.
Abstract (translated)
在这篇论文中，我们探讨了利用反向翻译（Back Translation，BT）作为半监督技术来增强英语-卢甘达语对的神经机器翻译（Neural Machine Translation, NMT）模型的应用，特别关注低资源语言所面临的挑战。我们的研究目的是展示如何通过从单语语料库生成合成数据来缓解双语数据稀缺问题。我们的方法包括使用公开可用和网络爬取的数据开发定制的NMT模型，并应用迭代和增量反向翻译技术。我们在多个小规模数据集中战略性地选择用于增量反向翻译的数据集，这是我们方法的一个新颖元素。 研究结果表明，在所有翻译方向上，英语-卢甘达语对的翻译性能都超过了先前的基准，提高了超过10个BLEU分数单位。此外，我们的评估包括SacreBLEU、ChrF2和TER等全面评估指标，从而为翻译质量提供了细致的理解。根据我们的研究结论，当精心策划的数据集被使用时，BT的有效性得到了确认，这不仅确立了新的性能基准，还展示了BT在增强低资源语言NMT模型方面的潜力。
URL
https://arxiv.org/abs/2505.02463
PDF
-
AI agents may be worth the hype but not the resources : An initial exploration of machine translation quality and costs in three language pairs in the legal and news domains
2025-05-02 20:02:13Vicent Briva Iglesias, Gokhan DogruAbstract
Large language models (LLMs) and multi-agent orchestration are touted as the next leap in machine translation (MT), but their benefits relative to conventional neural MT (NMT) remain unclear. This paper offers an empirical reality check. We benchmark five paradigms, Google Translate (strong NMT baseline), GPT-4o (general-purpose LLM), o1-preview (reasoning-enhanced LLM), and two GPT-4o-powered agentic workflows (sequential three-stage and iterative refinement), on test data drawn from a legal contract and news prose in three English-source pairs: Spanish, Catalan and Turkish. Automatic evaluation is performed with COMET, BLEU, chrF2 and TER; human evaluation is conducted with expert ratings of adequacy and fluency; efficiency with total input-plus-output token counts mapped to April 2025 pricing. Automatic scores still favour the mature NMT system, which ranks first in seven of twelve metric-language combinations; o1-preview ties or places second in most remaining cases, while both multi-agent workflows trail. Human evaluation reverses part of this narrative: o1-preview produces the most adequate and fluent output in five of six comparisons, and the iterative agent edges ahead once, indicating that reasoning layers capture semantic nuance undervalued by surface metrics. Yet these qualitative gains carry steep costs. The sequential agent consumes roughly five times, and the iterative agent fifteen times, the tokens used by NMT or single-pass LLMs. We advocate multidimensional, cost-aware evaluation protocols and highlight research directions that could tip the balance: leaner coordination strategies, selective agent activation, and hybrid pipelines combining single-pass LLMs with targeted agent intervention.
Abstract (translated)
大型语言模型（LLM）和多代理编排被吹捧为机器翻译（MT）的下一个飞跃，但它们相对于传统的神经网络机器翻译（NMT）的优势仍然不明确。本文提供了一个实证性的现实检验。我们在三种英语源对——西班牙语、加泰罗尼亚语和土耳其语——中选择测试数据来评估五种范式：Google Translate（强大的NMT基准）、GPT-4o（通用LLM）、o1-preview（增强推理的LLM）以及两种由GPT-4o驱动的代理工作流程（顺序三阶段和迭代细化）。自动评价使用COMET、BLEU、chrF2和TER进行；人工评估则通过专家对准确性和流畅性的评分来进行；效率则用输入加输出令牌总数映射到2025年4月的价格来衡量。在十二个指标语言组合中，成熟的NMT系统仍然在七个组合中得分最高；o1-preview在其余大部分情况下的排名紧随其后或并列第二，而两个多代理工作流程则落后。 人工评估部分逆转了这一结果：在六个比较中的五个中，o1-preview产生了最准确和流畅的输出，而在一次比较中迭代代理略胜一筹，这表明推理层捕捉到了表面指标所忽视的意义细微差别。然而，这些定性的提升伴随着高昂的成本：顺序代理消耗的令牌大约是NMT或单次通过LLM五倍左右，而迭代代理则为十五倍。 我们提倡采用多维度且考虑成本的评估协议，并指出可以改变这种平衡的研究方向，包括更精简的协调策略、选择性地激活代理以及结合单次通过LLM和有针对性的代理干预的混合管道。
URL
https://arxiv.org/abs/2505.01560
PDF
-
Improving Retrieval-Augmented Neural Machine Translation with Monolingual Data
2025-04-30 15:41:03Maxime Bouthors, Josep Crego, Fran\c{c}ois YvonAbstract
Conventional retrieval-augmented neural machine translation (RANMT) systems leverage bilingual corpora, e.g., translation memories (TMs). Yet, in many settings, in-domain monolingual target-side corpora are often available. This work explores ways to take advantage of such resources by retrieving relevant segments directly in the target language, based on a source-side query. For this, we design improved cross-lingual retrieval systems, trained with both sentence level and word-level matching objectives. In our experiments with two RANMT architectures, we first demonstrate the benefits of such cross-lingual objectives in a controlled setting, obtaining translation performances that surpass standard TM-based models. We then showcase our method on a real-world set-up, where the target monolingual resources far exceed the amount of parallel data and observe large improvements of our new techniques, which outperform both the baseline setting, and general-purpose cross-lingual retrievers.
Abstract (translated)
传统的检索增强神经机器翻译（RANMT）系统依赖于双语语料库，例如翻译记忆库（TMs）。然而，在许多情况下，目标语言中的单语领域内语料库通常是可以获取的。本研究探索了利用这些资源的方法，通过在目标语言中直接检索与源语言查询相关的段落来实现这一目标。为此，我们设计了改进的跨语言检索系统，并使用句子级别和词汇级别的匹配目标进行训练。 在两个RANMT架构上的实验中，我们在一个控制环境中首先展示了此类跨语言目标的优势，获得了超过标准TM基线模型的翻译性能。然后，我们在一个现实世界的设置下展示我们的方法，在该设置中目标单语资源远远超过了并行数据的数量，并观察到我们新技术带来的显著改进，这些技术在性能上超越了基线设置和通用型跨语言检索器。
URL
https://arxiv.org/abs/2504.21747
PDF
-
Calibrating Translation Decoding with Quality Estimation on LLMs
2025-04-26 22:38:47Di Wu, Yibin Lei, Christof MonzAbstract
Neural machine translation (NMT) systems typically employ maximum a posteriori (MAP) decoding to select the highest-scoring translation from the distribution mass. However, recent evidence highlights the inadequacy of MAP decoding, often resulting in low-quality or even pathological hypotheses -- the decoding objective is not aligned with real-world translation quality. This paper proposes calibrating hypothesis likelihoods with translation quality from a distribution view by directly optimizing their Pearson correlation -- thereby enhancing the effectiveness of translation decoding. With our method, translation on large language models (LLMs) improves substantially after limited training (2K instances per direction). This improvement is orthogonal to those achieved through supervised fine-tuning, leading to substantial gains across a broad range of metrics and human evaluations -- even when applied to top-performing translation-specialized LLMs fine-tuned on high-quality translation data, such as Tower, or when compared to recent preference optimization methods, like CPO. Moreover, the calibrated translation likelihood can directly serve as a strong proxy for translation quality, closely approximating or even surpassing some state-of-the-art translation quality estimation models, like CometKiwi. Lastly, our in-depth analysis demonstrates that calibration enhances the effectiveness of MAP decoding, thereby enabling greater efficiency in real-world deployment. The resulting state-of-the-art translation model, which covers 10 languages, along with the accompanying code and human evaluation data, has been released to the community: this https URL.
Abstract (translated)
神经机器翻译（NMT）系统通常采用最大后验概率（MAP）解码来从分布中选出最高分的译文。然而，近期的研究表明，MAP解码存在不足之处，常常导致低质量甚至病态的假设——其解码目标与现实世界的翻译质量不一致。本文提出了一种通过直接优化它们之间的皮尔逊相关性来校准假设可能性和翻译质量的方法，从而增强翻译解码的有效性。采用我们的方法，在有限训练（每方向2000个实例）后，大规模语言模型（LLMs）的翻译性能显著提升。这种改进与通过监督微调获得的进步相独立，并且在广泛的度量标准和人工评估中取得了实质性的增益——即使应用于经过高质量翻译数据精细调整的顶级专业翻译模型，如Tower或最近的偏好优化方法（CPO）。此外，校准后的翻译可能性可以直接作为翻译质量的强大代理，紧密接近甚至超越一些最先进的翻译质量估计模型，例如CometKiwi。最后，深入分析表明，校准可以增强MAP解码的有效性，在实际部署中实现更高的效率。我们发布的最先进的翻译模型涵盖了10种语言，并附有代码和人工评估数据，具体信息可访问以下链接：this https URL。
URL
https://arxiv.org/abs/2504.19044
PDF
-
Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo
2025-04-24 05:02:26Ocheme Anthony Ekle, Biswarup DasAbstract
In this study, we develop Neural Machine Translation (NMT) and Transformer-based transfer learning models for English-to-Igbo translation - a low-resource African language spoken by over 40 million people across Nigeria and West Africa. Our models are trained on a curated and benchmarked dataset compiled from Bible corpora, local news, Wikipedia articles, and Common Crawl, all verified by native language experts. We leverage Recurrent Neural Network (RNN) architectures, including Long Short-Term Memory (LSTM) and Gated Recurrent Units (GRU), enhanced with attention mechanisms to improve translation accuracy. To further enhance performance, we apply transfer learning using MarianNMT pre-trained models within the SimpleTransformers framework. Our RNN-based system achieves competitive results, closely matching existing English-Igbo benchmarks. With transfer learning, we observe a performance gain of +4.83 BLEU points, reaching an estimated translation accuracy of 70%. These findings highlight the effectiveness of combining RNNs with transfer learning to address the performance gap in low-resource language translation tasks.
Abstract (translated)
在这项研究中，我们为英伊博语（Igbo）翻译开发了神经机器翻译（NMT）和基于Transformer的迁移学习模型。伊博语是一种非洲低资源语言，在尼日利亚和西非地区有超过4000万人使用。我们的模型是在精心整理并基准测试过的数据集上进行训练的，该数据集包括从圣经文献、当地新闻、维基百科文章以及Common Crawl收集的数据，并且所有内容都经过了伊博语母语专家的验证。 我们利用循环神经网络（RNN）架构，包括长短期记忆网络（LSTM）和门控循环单元（GRU），并结合注意力机制以提高翻译准确性。为了进一步提升性能，我们在SimpleTransformers框架中应用了使用MarianNMT预训练模型进行迁移学习的方法。基于RNN的系统取得了与现有英伊博语基准相近的竞争性结果。通过引入迁移学习，我们观察到BLEU分数提高了4.83分，估计翻译准确率为70%。 这些发现表明，在低资源语言翻译任务中结合使用RNN和迁移学习能够有效弥补性能差距。
URL
https://arxiv.org/abs/2504.17252
PDF
-
VNJPTranslate: A comprehensive pipeline for Vietnamese-Japanese translation
2025-04-01 01:38:25Hoang Hai Phan, Nguyen Duc Minh Vu, Nam Dang PhuongAbstract
Neural Machine Translation (NMT) driven by Transformer architectures has advanced significantly, yet faces challenges with low-resource language pairs like Vietnamese-Japanese (Vi-Ja). Issues include sparse parallel data and handling linguistic/cultural nuances. Recent progress in Large Language Models (LLMs) with strong reasoning, often refined via Reinforcement Learning (RL), enables high-quality synthetic data generation. We introduce VNJPTranslate, a pipeline designed to systematically address the Vi-Ja translation task. It features a targeted data augmentation strategy using advanced LLMs with Chain-of-Thought prompting for challenging segments identified via corpus analysis. Subsequently, we employ efficient fine-tuning techniques (Unsloth with QLoRA) on a capable, low-parameter autoregressive model (specifically, a fine-tuned version of the 1.8B parameter Sailor model, which is based on the Qwen architecture) to create a practical and high-performing translation system. This integrated approach aims to improve Vi-Ja translation quality significantly over existing baselines.
Abstract (translated)
基于Transformer架构的神经机器翻译（NMT）技术取得了显著进展，但在处理像越南语-日语（Vi-Ja）这样的低资源语言对时仍然面临挑战。这些问题包括平行数据稀疏以及处理语言和文化细微差别的困难。近年来，在大型语言模型（LLMs）中取得的重大进步，这些模型通过强化学习（RL）增强了推理能力，并能够生成高质量的合成数据。我们在此介绍VNJPTranslate，这是一种专门设计来系统性解决Vi-Ja翻译任务的流水线方法。 VNJPTranslate采用了一种针对性的数据增强策略，该策略利用先进的LLMs并通过Chain-of-Thought提示技术针对语料库分析中识别出的难点部分进行处理。随后，我们使用高效的微调技术（如Unsloth和QLoRA）在一款功能强大且参数较少的自回归模型上进行了训练（具体而言，是在1.8B参数规模的Sailor模型基础上进行微调，该模型基于Qwen架构）。这种方法旨在构建一个实用而高性能的翻译系统，并显著提升Vi-Ja翻译的质量超过现有的基准方法。
URL
https://arxiv.org/abs/2504.00339
PDF
-
Is LLM the Silver Bullet to Low-Resource Languages Machine Translation?
2025-03-31 13:56:03Yewei Song, Lujun Li, Cedric Lothritz, Saad Ezzini, Lama Sleem, Niccolo Gentile, Radu State, Tegawend\'e F. Bissyand\'e, Jacques KleinAbstract
Low-Resource Languages (LRLs) present significant challenges in natural language processing due to their limited linguistic resources and underrepresentation in standard datasets. While recent advancements in Large Language Models (LLMs) and Neural Machine Translation (NMT) have substantially improved translation capabilities for high-resource languages, performance disparities persist for LRLs, particularly impacting privacy-sensitive and resource-constrained scenarios. This paper systematically evaluates the limitations of current LLMs across 200 languages using benchmarks such as FLORES-200. We also explore alternative data sources, including news articles and bilingual dictionaries, and demonstrate how knowledge distillation from large pre-trained models can significantly improve smaller LRL translations. Additionally, we investigate various fine-tuning strategies, revealing that incremental enhancements markedly reduce performance gaps on smaller LLMs.
Abstract (translated)
低资源语言（LRL）在自然语言处理中面临重大挑战，原因在于其有限的语料资源和在标准数据集中的代表性不足。尽管近年来大型语言模型（LLMs）和神经机器翻译（NMT）的进步大大提高了高资源语言的翻译能力，但在低资源语言上的性能差距仍然存在，特别是在隐私敏感性和资源受限的情境下影响尤为显著。本文系统地评估了当前LLMs在200种语言中的限制，并使用FLORES-200等基准测试进行衡量。我们还探索了替代数据源，包括新闻文章和双语词典，并展示了如何通过从大型预训练模型中提取知识来显著提高较小LRL的翻译质量。此外，本文研究了各种微调策略，揭示增量改进能够明显缩小在小型LLMs上的性能差距。
URL
https://arxiv.org/abs/2503.24102
PDF
-
Beyond Vanilla Fine-Tuning: Leveraging Multistage, Multilingual, and Domain-Specific Methods for Low-Resource Machine Translation
2025-03-28 16:30:28Sarubi Thillainathan, Songchen Yuan, En-Shiun Annie Lee, Sanath Jayasena, Surangika RanathungaAbstract
Fine-tuning multilingual sequence-to-sequence large language models (msLLMs) has shown promise in developing neural machine translation (NMT) systems for low-resource languages (LRLs). However, conventional single-stage fine-tuning methods struggle in extremely low-resource NMT settings, where training data is very limited. This paper contributes to artificial intelligence by proposing two approaches for adapting msLLMs in these challenging scenarios: (1) continual pre-training (CPT), where the msLLM is further trained with domain-specific monolingual data to compensate for the under-representation of LRLs, and (2) intermediate task transfer learning (ITTL), a method that fine-tunes the msLLM with both in-domain and out-of-domain parallel data to enhance its translation capabilities across various domains and tasks. As an application in engineering, these methods are implemented in NMT systems for Sinhala, Tamil, and English (six language pairs) in domain-specific, extremely low-resource settings (datasets containing fewer than 100,000 samples). Our experiments reveal that these approaches enhance translation performance by an average of +1.47 bilingual evaluation understudy (BLEU) score compared to the standard single-stage fine-tuning baseline across all translation directions. Additionally, a multi-model ensemble further improves performance by an additional BLEU score.
Abstract (translated)
针对低资源语言（LRLs）的神经机器翻译（NMT）系统，微调多语种序列到序列大规模语言模型（msLLMs）显示出了一定潜力。然而，在训练数据极为有限的极端低资源NMT环境中，传统的单阶段微调方法面临着挑战。本文通过提出两种适应msLLMs的方法来应对这些具有挑战性的场景：(1) 连续预训练（CPT），其中msLLM进一步使用特定领域的单语种数据进行训练以补偿LRLs的代表性不足；(2) 中间任务迁移学习（ITTL），这是一种利用域内和域外并行数据对msLLM进行微调的方法，旨在提高其在各种领域和任务中的翻译能力。作为工程应用的一部分，这些方法被应用于斯里兰卡语、泰米尔语与英语之间的六种语言组合的神经机器翻译系统，在特定领域的极端低资源环境下（包含少于10万样本的数据集）。实验结果显示，相较于标准单阶段微调基线模型，在所有翻译方向上，这些建议的方法提升了平均+1.47分的双语评估研究工具（BLEU）评分。此外，通过多模型集成进一步提高了性能，并额外增加了BLEU分数。
URL
https://arxiv.org/abs/2503.22582
PDF
-
Contextual Cues in Machine Translation: Investigating the Potential of Multi-Source Input Strategies in LLMs and NMT Systems
2025-03-10 11:23:44Lia Shahnazaryan, Patrick Simianer, Joern WuebkerAbstract
We explore the impact of multi-source input strategies on machine translation (MT) quality, comparing GPT-4o, a large language model (LLM), with a traditional multilingual neural machine translation (NMT) system. Using intermediate language translations as contextual cues, we evaluate their effectiveness in enhancing English and Chinese translations into Portuguese. Results suggest that contextual information significantly improves translation quality for domain-specific datasets and potentially for linguistically distant language pairs, with diminishing returns observed in benchmarks with high linguistic variability. Additionally, we demonstrate that shallow fusion, a multi-source approach we apply within the NMT system, shows improved results when using high-resource languages as context for other translation pairs, highlighting the importance of strategic context language selection.
Abstract (translated)
我们探讨了多源输入策略对机器翻译（MT）质量的影响，比较了一个大型语言模型（LLM）GPT-4o与传统的多语种神经机器翻译（NMT）系统。通过使用中间语言的翻译作为上下文线索，我们评估了这种方法在增强英语和汉语到葡萄牙语翻译效果中的有效性。结果显示，上下文信息显著提高了领域特定数据集的翻译质量，并且对于语言差异较大的语言对也具有潜在的好处，在语言变异性较高的基准测试中则表现出收益递减的现象。此外，我们还展示了浅层融合方法（我们在NMT系统内应用的一种多源策略）在使用高资源语言作为其他翻译任务上下文时效果更佳，强调了战略性选择上下文语言的重要性。
URL
https://arxiv.org/abs/2503.07195
PDF
-
Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translation
2025-03-09 12:54:05Yingfeng Luo, Tong Zheng, Yongyu Mu, Bei Li, Qinghong Zhang, Yongqi Gao, Ziqiang Xu, Peinan Feng, Xiaoqian Liu, Tong Xiao, Jingbo ZhuAbstract
The field of neural machine translation (NMT) has changed with the advent of large language models (LLMs). Much of the recent emphasis in natural language processing (NLP) has been on modeling machine translation and many other problems using a single pre-trained Transformer decoder, while encoder-decoder architectures, which were the standard in earlier NMT models, have received relatively less attention. In this paper, we explore translation models that are universal, efficient, and easy to optimize, by marrying the world of LLMs with the world of NMT. We apply LLMs to NMT encoding and leave the NMT decoder unchanged. We also develop methods for adapting LLMs to work better with the NMT decoder. Furthermore, we construct a new dataset involving multiple tasks to assess how well the machine translation system generalizes across various tasks. Evaluations on the WMT and our datasets show that results using our method match or surpass a range of baselines in terms of translation quality, but achieve $2.4 \sim 6.5 \times$ inference speedups and a $75\%$ reduction in the memory footprint of the KV cache. It also demonstrates strong generalization across a variety of translation-related tasks.
Abstract (translated)
神经机器翻译（NMT）领域随着大型语言模型（LLMs）的出现发生了变化。最近自然语言处理（NLP）领域的重点是使用单个预训练的Transformer解码器来建模机器翻译及其他许多问题，而早期NMT模型中使用的编码器-解码器架构则相对较少受到关注。在本文中，我们探索了一种将LLM的世界与NMT的世界结合的方法，以创建通用、高效且易于优化的翻译模型。我们将LLMs应用于NMT编码，并保持原有的NMT解码器不变。同时，我们也开发了适应方法来改进LLMs使其更好地配合NMT解码器工作。此外，我们构建了一个包含多个任务的新数据集，用于评估机器翻译系统在各种任务中的泛化能力。我们在WMT和我们的数据集上进行了测试，结果显示使用我们提出的方法所得到的翻译质量与一系列基线方法相比不相上下甚至更优，但实现了2.4至6.5倍的速度提升以及KV缓存内存占用减少了75%。此外，该方法还展示了在各种翻译相关任务上的强大泛化能力。
URL
https://arxiv.org/abs/2503.06594
PDF
