Abstract
Hypothesis. Artificial general intelligence is, at its core, a compression problem. Effective compression demands resonance: deep learning scales best when its architecture aligns with the fundamental structure of the data. These are the fundamental principles. Yet, modern vision architectures have strayed from these truths: visual signals are highly redundant, while discriminative information, the surprise, is sparse. Current models process dense pixel grids uniformly, wasting vast compute on static background rather than focusing on the predictive residuals that define motion and meaning. We argue that to solve visual understanding, we must align our architectures with the information-theoretic principles of video, i.e., Codecs. Method. OneVision-Encoder encodes video by compressing predictive visual structure into semantic meaning. By adopting Codec Patchification, OV-Encoder abandons uniform computation to focus exclusively on the 3.1%-25% of regions rich in signal entropy. To unify spatial and temporal reasoning under irregular token layouts, OneVision-Encoder employs a shared 3D RoPE and is trained with a large-scale cluster discrimination objective over more than one million semantic concepts, jointly capturing object permanence and motion dynamics. Evidence. The results validate our core hypothesis: efficiency and accuracy are not a trade-off; they are positively correlated. When integrated into LLM, it consistently outperforms strong vision backbones such as Qwen3-ViT and SigLIP2 across 16 image, video, and document understanding benchmarks, despite using substantially fewer visual tokens and pretraining data. Notably, on video understanding tasks, OV-Encoder achieves an average improvement of 4.1% over Qwen3-ViT. Codec-aligned, patch-level sparsity is a foundational principle, enabling OV-Encoder as a scalable engine for next-generation visual generalists.
Abstract (translated)
假设。人工智能的通用智能在根本上是一个压缩问题。有效的压缩需要共振：深度学习在架构与数据的基本结构相匹配时表现最佳。这些是基本原则。然而，现代视觉架构已经偏离了这一真理：视觉信号高度冗余，而判别信息（即惊喜）则非常稀疏。当前模型对密集的像素网格进行均匀处理，浪费了大量的计算资源来处理静态背景，而不是专注于定义运动和意义的预测残差上。我们认为，要解决视觉理解问题，我们必须使架构与视频的信息论原理——编码器-解码器系统（Codec）相一致。 方法。OneVision-Encoder 通过将预测性的视觉结构压缩为语义含义来对视频进行编码。通过采用 Codec 补丁化技术，OV-Encoder 放弃了均匀计算，并专注于信号熵丰富的区域，这些区域占总区域的3.1%-25%之间。为了在不规则令牌布局下统一空间和时间推理，OneVision-Encoder 使用共享的三维RoPE（旋转位置嵌入），并通过超过一百万个语义概念的大规模集群判别目标进行训练，同时捕捉对象持久性和运动动态。 证据。结果验证了我们的核心假设：效率与准确度并不是权衡的关系；它们是正相关的。当集成到大型语言模型中时，OneVision-Encoder 在16个图像、视频和文档理解基准测试上始终优于强大的视觉骨干网络，如Qwen3-ViT和SigLIP2，尽管使用了更少的视觉令牌和预训练数据。值得注意的是，在视频理解任务中，OV-Encoder 相对于 Qwen3-ViT 平均提高了4.1%。 Codec 对齐、补丁级稀疏性是基本原则，使 OV-Encoder 成为下一代视觉通用智能的基础引擎。
URL
https://arxiv.org/abs/2602.08683
