Abstract
We propose an ensemble model for predictingthe lexical complexity of words and multiwordexpressions (MWEs). The model receives asinput a sentence with a target word or MWEand outputs its complexity score. Given thata key challenge with this task is the limitedsize of annotated data, our model relies onpretrained contextual representations from dif-ferent state-of-the-art transformer-based lan-guage models (i.e., BERT and RoBERTa), andon a variety of training methods for further en-hancing model generalization and robustness:multi-step fine-tuning and multi-task learning,and adversarial training. Additionally, we pro-pose to enrich contextual representations byadding hand-crafted features during training.Our model achieved competitive results andranked among the top-10 systems in both sub-tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2105.05535
