Abstract
We propose LAPIG, a language guided projector image generation method with surface adaptation and stylization. LAPIG consists of a projector-camera system and a target textured projection surface. LAPIG takes the user text prompt as input and aims to transform the surface style using the projector. LAPIG's key challenge is that due to the projector's physical brightness limitation and the surface texture, the viewer's perceived projection may suffer from color saturation and artifacts in both dark and bright regions, such that even with the state-of-the-art projector compensation techniques, the viewer may see clear surface texture-related artifacts. Therefore, how to generate a projector image that follows the user's instruction while also displaying minimum surface artifacts is an open problem. To address this issue, we propose projection surface adaptation (PSA) that can generate compensable surface stylization. We first train two networks to simulate the projector compensation and project-and-capture processes, this allows us to find a satisfactory projector image without real project-and-capture and utilize gradient descent for fast convergence. Then, we design content and saturation losses to guide the projector image generation, such that the generated image shows no clearly perceivable artifacts when projected. Finally, the generated image is projected for visually pleasing surface style morphing effects. The source code and video are available on the project page: this https URL.
Abstract (translated)
我们提出了一种名为LAPIG的语言引导投影仪图像生成方法，该方法具备表面适应和风格化功能。LAPIG系统由一个投影仪-摄像机系统和一个目标纹理投影表面组成。LAPIG接收用户文本提示作为输入，并旨在使用投影仪改变表面样式。LAPIG面临的关键挑战是由于投影仪物理亮度限制以及表面纹理的存在，观众所感知到的投影图像在暗区与亮区可能会遭受颜色饱和度降低和失真现象，即使采用最先进的投影补偿技术也无法完全消除这些问题，使得观者仍能察觉到明显的表面纹理相关瑕疵。因此，在遵循用户指令的同时生成一个能够最小化表面瑕疵的投影仪图像成为了亟待解决的问题。 为了解决这个问题，我们提出了一种名为“投影表面适应”（PSA）的技术，该技术可以生成可补偿的表面风格化效果。首先，我们训练了两个网络来模拟投影补偿和投射捕捉过程，这使我们在不进行实际投射捕捉的情况下也能找到满意的投影仪图像，并利用梯度下降法实现快速收敛。随后，我们设计了内容损失和饱和度损失以指导投影仪图像的生成，从而使得生成的图像在被投射时不会产生明显的可感知瑕疵。最终，所生成的图像将被用于视觉上吸引人的表面样式变形效果。 项目的源代码和演示视频可在以下链接获取：[请在此处插入实际URL]
URL
https://arxiv.org/abs/2503.12173
