Paper Reading AI Learner

Distilling Text into Circuits

2023-01-25 13:56:34
Vincent Wang-Mascianica, Jonathon Liu, Bob Coecke

Abstract

This paper concerns the structure of meanings within natural language. Earlier, a framework named DisCoCirc was sketched that (1) is compositional and distributional (a.k.a. vectorial); (2) applies to general text; (3) captures linguistic `connections' between meanings (cf. grammar) (4) updates word meanings as text progresses; (5) structures sentence types; (6) accommodates ambiguity. Here, we realise DisCoCirc for a substantial fragment of English. When passing to DisCoCirc's text circuits, some `grammatical bureaucracy' is eliminated, that is, DisCoCirc displays a significant degree of (7) inter- and intra-language independence. That is, e.g., independence from word-order conventions that differ across languages, and independence from choices like many short sentences vs. few long sentences. This inter-language independence means our text circuits should carry over to other languages, unlike the language-specific typings of categorial grammars. Hence, text circuits are a lean structure for the `actual substance of text', that is, the inner-workings of meanings within text across several layers of expressiveness (cf. words, sentences, text), and may capture that what is truly universal beneath grammar. The elimination of grammatical bureaucracy also explains why DisCoCirc: (8) applies beyond language, e.g. to spatial, visual and other cognitive modes. While humans could not verbally communicate in terms of text circuits, machines can. We first define a `hybrid grammar' for a fragment of English, i.e. a purpose-built, minimal grammatical formalism needed to obtain text circuits. We then detail a translation process such that all text generated by this grammar yields a text circuit. Conversely, for any text circuit obtained by freely composing the generators, there exists a text (with hybrid grammar) that gives rise to it. Hence: (9) text circuits are generative for text.

Abstract (translated)

这篇文章讨论了自然语言中意义结构的问题。之前,我们概述了一个框架名为DisCoCirc,它的特点是(1)是Compositional and Distributional(也就是Vectorial),(2)适用于一般文本,(3)能够捕捉语言中的“连接关系”(类似于语法),(4)随着文本的进展更新单词含义,(5)构建句子类型,(6)适应歧义。在这里,我们实现了DisCoCirc对一个大量英语文本的部分实现。当进入DisCoCirc的文本电路时,一些“语法官僚主义”被消除,也就是DisCoCirc表现出(7)跨语言和内部语言独立性。例如,独立性于不同语言中的词序习惯,以及类似于许多短句子和小句子的选择。这种跨语言独立性意味着我们的文本电路应该扩展到其他语言,而不像语法特定的词汇语法 typing。因此,文本电路是一种 lean 的结构,适用于“实际文本内容”即表达层数多层的含义内部运作(类似于单词、句子和文本),并可能捕捉到语法之下真正普遍的机制。消除语法官僚主义还解释了为什么DisCoCirc:(8)适用于超越语言,例如空间、视觉和其他认知模式。虽然人类无法在文本电路中口头沟通,但机器可以。我们首先定义了一个“混合语法”为英语文本的片段,即需要构建的最小语法形式化,以获得文本电路。然后我们详细描述了一种翻译过程,以便所有由该语法生成的文本生成文本电路。反之亦然,对于任何由自由组合生成器的文本电路,存在一个文本(具有混合语法)导致它。因此:(9)文本电路是文本生成的基础。

URL

https://arxiv.org/abs/2301.10595

PDF

https://arxiv.org/pdf/2301.10595.pdf


Tags
3D Action Action_Localization Action_Recognition Activity Adversarial Agent Attention Autonomous Bert Boundary_Detection Caption Chat Classification CNN Compressive_Sensing Contour Contrastive_Learning Deep_Learning Denoising Detection Dialog Diffusion Drone Dynamic_Memory_Network Edge_Detection Embedding Embodied Emotion Enhancement Face Face_Detection Face_Recognition Facial_Landmark Few-Shot Gait_Recognition GAN Gaze_Estimation Gesture Gradient_Descent Handwriting Human_Parsing Image_Caption Image_Classification Image_Compression Image_Enhancement Image_Generation Image_Matting Image_Retrieval Inference Inpainting Intelligent_Chip Knowledge Knowledge_Graph Language_Model Matching Medical Memory_Networks Multi_Modal Multi_Task NAS NMT Object_Detection Object_Tracking OCR Ontology Optical_Character Optical_Flow Optimization Person_Re-identification Point_Cloud Portrait_Generation Pose Pose_Estimation Prediction QA Quantitative Quantitative_Finance Quantization Re-identification Recognition Recommendation Reconstruction Regularization Reinforcement_Learning Relation Relation_Extraction Represenation Represenation_Learning Restoration Review RNN Salient Scene_Classification Scene_Generation Scene_Parsing Scene_Text Segmentation Self-Supervised Semantic_Instance_Segmentation Semantic_Segmentation Semi_Global Semi_Supervised Sence_graph Sentiment Sentiment_Classification Sketch SLAM Sparse Speech Speech_Recognition Style_Transfer Summarization Super_Resolution Surveillance Survey Text_Classification Text_Generation Tracking Transfer_Learning Transformer Unsupervised Video_Caption Video_Classification Video_Indexing Video_Prediction Video_Retrieval Visual_Relation VQA Weakly_Supervised Zero-Shot