Abstract
Traditional sentence embedding models encode sentences into vector representations to capture useful properties such as the semantic similarity between sentences. However, in addition to similarity, sentence semantics can also be interpreted via compositional operations such as sentence fusion or difference. It is unclear whether the compositional semantics of sentences can be directly reflected as compositional operations in the embedding space. To more effectively bridge the continuous embedding and discrete text spaces, we explore the plausibility of incorporating various compositional properties into the sentence embedding space that allows us to interpret embedding transformations as compositional sentence operations. We propose InterSent, an end-to-end framework for learning interpretable sentence embeddings that supports compositional sentence operations in the embedding space. Our method optimizes operator networks and a bottleneck encoder-decoder model to produce meaningful and interpretable sentence embeddings. Experimental results demonstrate that our method significantly improves the interpretability of sentence embeddings on four textual generation tasks over existing approaches while maintaining strong performance on traditional semantic similarity tasks.
Abstract (translated)
传统的句子嵌入模型将句子编码为向量表示，以捕捉句子之间的有用性质，例如句子语义相似性。然而，除了相似性，句子语义还可以通过组合性操作例如句子融合或差异来解释。目前尚不清楚句子组合性语义是否可以在嵌入空间中直接反映。为了更有效地连接连续嵌入和离散文本空间，我们探索将各种组合性属性嵌入句子嵌入空间的可能性，以便将嵌入变换解释为组合性句子操作。我们提出了InterSent，一个端到端的框架，以学习可解释的句子嵌入，支持在嵌入空间中的组合性句子操作。我们的方法和优化了操作网络和瓶颈编码解码模型，以产生有意义且可解释的句子嵌入。实验结果显示，我们的方法和现有的方法在四个文本生成任务上相比，在传统的语义相似任务上表现出显著提高，同时保持了强大的传统语义相似性任务表现。
URL
https://arxiv.org/abs/2305.14599
