Abstract
This study explores four methods of generating paraphrases in Malayalam, utilizing resources available for English paraphrasing and pre-trained Neural Machine Translation (NMT) models. We evaluate the resulting paraphrases using both automated metrics, such as BLEU, METEOR, and cosine similarity, as well as human annotation. Our findings suggest that automated evaluation measures may not be fully appropriate for Malayalam, as they do not consistently align with human judgment. This discrepancy underscores the need for more nuanced paraphrase evaluation approaches especially for highly agglutinative languages.
Abstract (translated)
这项研究探讨了在 Malayalam 中生成paraphrases的四种方法，利用了可用于英语paraphrasing的现有资源和预训练的 Neural Machine Translation（NMT）模型。我们使用自动指标（如 BLEU、METEOR 和余弦相似性）和人类注释来评估所得的paraphrases。我们的发现表明，自动评估措施可能并不完全适用于 Malayalam，因为它们并不始终与人类判断相一致。这一差异突显了在高度黏着性语言中，需要更加细微的paraphrase评估方法。
URL
https://arxiv.org/abs/2401.17827
