Abstract
We propose a new fast word embedding technique using hash functions. The method is a derandomization of a new type of random projections: By disregarding the classic constraint used in designing random projections (i.e., preserving pairwise distances in a particular normed space), our solution exploits extremely sparse non-negative random projections. Our experiments show that the proposed method can achieve competitive results, comparable to neural embedding learning techniques, however, with only a fraction of the computational complexity of these methods. While the proposed derandomization enhances the computational and space complexity of our method, the possibility of applying weighting methods such as positive pointwise mutual information (PPMI) to our models after their construction (and at a reduced dimensionality) imparts a high discriminatory power to the resulting embeddings. Obviously, this method comes with other known benefits of random projection-based techniques such as ease of update.
Abstract (translated)
我们提出了一种使用散列函数的新的快速字嵌入技术。该方法是新型随机投影的去随机化：通过忽略用于设计随机投影的经典约束（即，保留特定赋范空间中的成对距离），我们的解决方案利用极稀疏的非负随机投影。我们的实验表明，与神经嵌入学习技术相比，所提出的方法可以获得有竞争力的结果，然而，这些方法的计算复杂性只有一小部分。虽然所提出的去随机化增强了我们方法的计算和空间复杂性，但是在构造之后（以及在降低的维度上）将加权方法（例如正向点互信息（PPMI））应用于我们的模型的可能性赋予结果高的辨别力。的嵌入。显然，这种方法具有基于随机投影的技术的其他已知优点，例如易于更新。
URL
https://arxiv.org/abs/1705.04253
