Abstract
Moire patterns, unwanted color artifacts in images and videos, arise from the interference between spatially high-frequency scene contents and the spatial discrete sampling of digital cameras. Existing demoireing methods primarily rely on single-camera image/video processing, which faces two critical challenges: 1) distinguishing moire patterns from visually similar real textures, and 2) preserving tonal consistency and temporal coherence while removing moire artifacts. To address these issues, we propose a dual-camera framework that captures synchronized videos of the same scene: one in focus (retaining high-quality textures but may exhibit moire patterns) and one defocused (with significantly reduced moire patterns but blurred textures). We use the defocused video to help distinguish moire patterns from real texture, so as to guide the demoireing of the focused video. We propose a frame-wise demoireing pipeline, which begins with an optical flow based alignment step to address any discrepancies in displacement and occlusion between the focused and defocused frames. Then, we leverage the aligned defocused frame to guide the demoireing of the focused frame using a multi-scale CNN and a multi-dimensional training loss. To maintain tonal and temporal consistency, our final step involves a joint bilateral filter to leverage the demoireing result from the CNN as the guide to filter the input focused frame to obtain the final output. Experimental results demonstrate that our proposed framework largely outperforms state-of-the-art image and video demoireing methods.
Abstract (translated)
摩尔纹是图像和视频中不需要的颜色伪影，它们源自空间高频场景内容与数字相机空间离散采样之间的干扰。现有的去摩尔纹方法主要依赖于单摄像头的图像或视频处理，面临着两个关键挑战：1）区分摩尔纹与视觉上相似的真实纹理；2）在去除摩尔纹的同时保持色调一致性和时间连贯性。为了解决这些问题，我们提出了一种双摄像头框架，该框架捕捉同一场景同步录制的两段视频：一段是聚焦模式（保留高质量的纹理但可能显示摩尔纹），另一段是非聚焦模式（显著减少摩尔纹但会模糊纹理）。利用非聚焦视频来帮助区分摩尔纹与真实纹理，并以此指导对聚焦视频进行去摩尔处理。我们提出了一种帧级去摩尔流水线，首先采用基于光学流的对齐步骤解决聚焦和非聚焦帧在位移和遮挡上的不一致问题。随后，利用已对齐的非聚焦帧来引导聚焦帧的去摩尔处理，通过多尺度卷积神经网络（CNN）及多维度训练损失实现这一过程。为了保持色调和时间的一致性，我们的最终步骤涉及联合双边滤波器，利用CNN输出的去摩尔结果作为指导过滤输入的聚焦视频，以获得最终输出。 实验结果显示，我们提出的框架在图像和视频去摩尔处理方法中远远超过了当前的最佳水平。
URL
https://arxiv.org/abs/2508.03449
