Abstract
Commonly used image tokenizers produce a 2D grid of spatially arranged tokens. In contrast, so-called 1D image tokenizers represent images as highly compressed one-dimensional sequences of as few as 32 discrete tokens. We find that the high degree of compression achieved by a 1D tokenizer with vector quantization enables image editing and generative capabilities through heuristic manipulation of tokens, demonstrating that even very crude manipulations -- such as copying and replacing tokens between latent representations of images -- enable fine-grained image editing by transferring appearance and semantic attributes. Motivated by the expressivity of the 1D tokenizer's latent space, we construct an image generation pipeline leveraging gradient-based test-time optimization of tokens with plug-and-play loss functions such as reconstruction or CLIP similarity. Our approach is demonstrated for inpainting and text-guided image editing use cases, and can generate diverse and realistic samples without requiring training of any generative model.
Abstract (translated)
常用的图像标记化方法会产生一个二维的、空间排列的标记网格。相比之下，所谓的1D图像标记器将图像表示为高度压缩的一维序列，最少仅包含32个离散标记。我们发现，通过向量量化实现的高度压缩能力使得基于1D标记器进行图像编辑和生成功能成为可能，并且通过对标记进行启发式操作就能实现这一点。即使是最粗糙的操作——如在不同图像的潜在表示之间复制和替换标记——也能使精细的图像编辑成为现实，包括传输外观和语义属性。 受到1D标记器隐空间表达能力的启发，我们构建了一个基于梯度测试时间优化标记（使用可插拔损失函数如重构或CLIP相似性）的图像生成流水线。我们的方法在图像修复和文本引导的图像编辑场景中得到展示，并且可以生成多样而逼真的样本，无需训练任何生成模型。
URL
https://arxiv.org/abs/2506.08257
