Abstract
Adverse weather conditions cause diverse and complex degradation patterns, driving the development of All-in-One (AiO) models. However, recent AiO solutions still struggle to capture diverse degradations, since global filtering methods like direct operations on the frequency domain fail to handle highly variable and localized distortions. To address these issue, we propose Spectral-based Spatial Grouping Transformer (SSGformer), a novel approach that leverages spectral decomposition and group-wise attention for multi-weather image restoration. SSGformer decomposes images into high-frequency edge features using conventional edge detection and low-frequency information via Singular Value Decomposition. We utilize multi-head linear attention to effectively model the relationship between these features. The fused features are integrated with the input to generate a grouping-mask that clusters regions based on the spatial similarity and image texture. To fully leverage this mask, we introduce a group-wise attention mechanism, enabling robust adverse weather removal and ensuring consistent performance across diverse weather conditions. We also propose a Spatial Grouping Transformer Block that uses both channel attention and spatial attention, effectively balancing feature-wise relationships and spatial dependencies. Extensive experiments show the superiority of our approach, validating its effectiveness in handling the varied and intricate adverse weather degradations.
Abstract (translated)
恶劣的天气条件会导致各种复杂且多样的图像退化模式，这推动了All-in-One（AiO）模型的发展。然而，最近的AiO解决方案仍然难以捕捉到这些多样化的退化情况，因为像频域直接操作这样的全局滤波方法无法处理高度变化和局部分布的失真。为了解决这些问题，我们提出了一种基于谱分解的空间分组变换器(SSGformer)，这是一种新颖的方法，利用谱分解和分组注意力机制来进行多天气图像恢复。 SSGformer通过传统的边缘检测将图像分解成高频边缘特征，并使用奇异值分解提取低频信息。我们采用多头线性注意机制有效地建模这些特征之间的关系。融合后的特征与输入结合生成一个分组掩码，根据空间相似性和图像纹理对区域进行聚类。为了充分利用这个掩码，我们引入了一种基于分组的注意力机制，从而实现稳健的恶劣天气去除，并确保在各种天气条件下的一致性能。 此外，我们还提出了一种空间分组变换器块（Spatial Grouping Transformer Block），该模块结合了通道注意和空间注意，有效地平衡了特征关系与空间依赖性。广泛的实验表明，我们的方法优于现有技术，在处理复杂且多样的恶劣天气退化方面表现出色。
URL
https://arxiv.org/abs/2507.22498
