Compressive_Sensing
-
Compressive Imaging Reconstruction via Tensor Decomposed Multi-Resolution Grid Encoding
2025-07-10 12:36:20Zhenyu Jin, Yisi Luo, Xile Zhao, Deyu MengAbstract
Compressive imaging (CI) reconstruction, such as snapshot compressive imaging (SCI) and compressive sensing magnetic resonance imaging (MRI), aims to recover high-dimensional images from low-dimensional compressed measurements. This process critically relies on learning an accurate representation of the underlying high-dimensional image. However, existing unsupervised representations may struggle to achieve a desired balance between representation ability and efficiency. To overcome this limitation, we propose Tensor Decomposed multi-resolution Grid encoding (GridTD), an unsupervised continuous representation framework for CI reconstruction. GridTD optimizes a lightweight neural network and the input tensor decomposition model whose parameters are learned via multi-resolution hash grid encoding. It inherently enjoys the hierarchical modeling ability of multi-resolution grid encoding and the compactness of tensor decomposition, enabling effective and efficient reconstruction of high-dimensional images. Theoretical analyses for the algorithm's Lipschitz property, generalization error bound, and fixed-point convergence reveal the intrinsic superiority of GridTD as compared with existing continuous representation models. Extensive experiments across diverse CI tasks, including video SCI, spectral SCI, and compressive dynamic MRI reconstruction, consistently demonstrate the superiority of GridTD over existing methods, positioning GridTD as a versatile and state-of-the-art CI reconstruction method.
Abstract (translated)
压缩成像（CI）重建技术，如快照压缩成像（SCI）和压缩感知磁共振成像（MRI），旨在从低维压缩测量中恢复高维度图像。这一过程关键在于准确学习底层高维图像的表示形式。然而，现有的无监督表示方法可能难以在表征能力和效率之间取得理想的平衡。为克服这一限制，我们提出了一种针对CI重建的无监督连续表示框架——张量分解多分辨率网格编码（GridTD）。GridTD通过多层次哈希网格编码优化轻量级神经网络和输入张量分解模型，并从中学习参数。它天然具备多层次网格编码的层次建模能力和张量分解的紧凑性，能够有效地实现高维图像重建。 算法的Lipschitz性质、泛化误差界限以及固定点收敛性的理论分析揭示了GridTD与现有连续表示模型相比所具有的内在优越性。在包括视频SCI、光谱SCI和压缩动态MRI重建等多样的CI任务中进行的广泛实验，一致地证明了GridTD优于现有的方法，并将GridTD定位为一种通用且先进的CI重建技术。
URL
https://arxiv.org/abs/2507.07707
PDF
-
Robust Node Localization for Rough and Extreme Deployment Environments
Abstract
Many applications have been identified which require the deployment of large-scale low-power wireless sensor networks. Some of the deployment environments, however, impose harsh operation conditions due to intense cross-technology interference, extreme weather conditions (heavy rainfall, excessive heat, etc.), or rough motion, thereby affecting the quality and predictability of the wireless links the nodes establish. In localization tasks, these conditions often lead to significant errors in estimating the position of target nodes. Motivated by the practical deployments of sensors on the surface of different water bodies, we address the problem of identifying susceptible nodes and robustly estimating their positions. We formulate these tasks as a compressive sensing problem and propose algorithms for both node identification and robust estimation. Additionally, we design an optimal anchor configuration to maximize the robustness of the position estimation task. Our numerical results and comparisons with competitive methods demonstrate that the proposed algorithms achieve both objectives with a modest number of anchors. Since our method relies only on target-to-anchor distances, it is broadly applicable and yields resilient, robust localization.
Abstract (translated)
许多应用场景已确定需要部署大规模低功耗无线传感器网络。然而，一些部署环境由于存在强烈的跨技术干扰、极端天气条件（如暴雨和高温）或剧烈的运动等因素，给节点建立无线链路的质量和可预测性带来了挑战，从而导致定位任务中目标节点位置估计出现显著误差。 受在不同水体表面部署传感器的实际应用启发，我们致力于识别易受影响的节点并稳健地估算其位置。我们将这些问题转化为压缩感知问题，并提出了用于节点识别和稳健估计的算法。此外，我们还设计了一种最优锚点配置方案以最大化定位任务的鲁棒性。我们的数值结果与竞争方法相比表明，所提出的算法能够使用少量锚点实现这两个目标。 由于本方法仅依赖于目标到锚点的距离信息，它具有广泛适用性和提供稳健、精确的定位能力的特点。
URL
https://arxiv.org/abs/2507.03856
PDF
-
Zero-Shot Solving of Imaging Inverse Problems via Noise-Refined Likelihood Guided Diffusion Models
2025-06-16 11:56:50Zhen Wang, Hongyi Liu, Zhihui WeiAbstract
Diffusion models have achieved remarkable success in imaging inverse problems owing to their powerful generative capabilities. However, existing approaches typically rely on models trained for specific degradation types, limiting their generalizability to various degradation scenarios. To address this limitation, we propose a zero-shot framework capable of handling various imaging inverse problems without model retraining. We introduce a likelihood-guided noise refinement mechanism that derives a closed-form approximation of the likelihood score, simplifying score estimation and avoiding expensive gradient computations. This estimated score is subsequently utilized to refine the model-predicted noise, thereby better aligning the restoration process with the generative framework of diffusion models. In addition, we integrate the Denoising Diffusion Implicit Models (DDIM) sampling strategy to further improve inference efficiency. The proposed mechanism can be applied to both optimization-based and sampling-based schemes, providing an effective and flexible zero-shot solution for imaging inverse problems. Extensive experiments demonstrate that our method achieves superior performance across multiple inverse problems, particularly in compressive sensing, delivering high-quality reconstructions even at an extremely low sampling rate (5%).
Abstract (translated)
扩散模型在成像逆问题中取得了显著的成功，这得益于其强大的生成能力。然而，现有的方法通常依赖于针对特定退化类型训练的模型，从而限制了它们在各种退化场景中的泛化能力。为了解决这一局限性，我们提出了一种零样本框架，该框架能够在不重新训练模型的情况下处理多种成像逆问题。我们引入了一个基于似然性的噪声精炼机制，它可以推导出一个闭式形式的似然分数近似，简化了分数估计过程，并避免了昂贵的梯度计算。这个估计出来的分数随后被用来改进模型预测的噪声，从而更好地使恢复过程与扩散模型的生成框架相一致。此外，我们将去噪扩散隐式模型（DDIM）采样策略集成进来，进一步提高了推理效率。所提出的机制既可以应用于基于优化的方法也可以用于基于抽样的方法，为成像逆问题提供了一个有效且灵活的零样本解决方案。广泛的实验表明，我们的方法在多个逆问题上实现了优越的表现，特别是在压缩感知领域，在极低的采样率（5%）下也能实现高质量的重建。
URL
https://arxiv.org/abs/2506.13391
PDF
-
RadioDUN: A Physics-Inspired Deep Unfolding Network for Radio Map Estimation
2025-06-10 03:46:20Taiqin Chen, Zikun Zhou, Zheng Fang, Wenzhen Zou, Kanjun Liu, Ke Chen, Yongbing Zhang, Yaowei WangAbstract
The radio map represents the spatial distribution of spectrum resources within a region, supporting efficient resource allocation and interference mitigation. However, it is difficult to construct a dense radio map as a limited number of samples can be measured in practical scenarios. While existing works have used deep learning to estimate dense radio maps from sparse samples, they are hard to integrate with the physical characteristics of the radio map. To address this challenge, we cast radio map estimation as the sparse signal recovery problem. A physical propagation model is further incorporated to decompose the problem into multiple factor optimization sub-problems, thereby reducing recovery complexity. Inspired by the existing compressive sensing methods, we propose the Radio Deep Unfolding Network (RadioDUN) to unfold the optimization process, achieving adaptive parameter adjusting and prior fitting in a learnable manner. To account for the radio propagation characteristics, we develop a dynamic reweighting module (DRM) to adaptively model the importance of each factor for the radio map. Inspired by the shadowing factor in the physical propagation model, we integrate obstacle-related factors to express the obstacle-induced signal stochastic decay. The shadowing loss is further designed to constrain the factor prediction and act as a supplementary supervised objective, which enhances the performance of RadioDUN. Extensive experiments have been conducted to demonstrate that the proposed method outperforms the state-of-the-art methods. Our code will be made publicly available upon publication.
Abstract (translated)
无线电图表示了某个区域内频谱资源的空间分布，支持高效的资源分配和干扰抑制。然而，在实际场景中由于可测量的样本数量有限，构建密集型无线电图非常困难。现有的研究工作使用深度学习从稀疏样本估算出密集型无线电图，但难以与无线电图的物理特性相结合。为了解决这一挑战，我们将无线电图估计视为稀疏信号恢复问题。通过进一步结合物理传播模型，将该问题分解成多个因子优化子问题，从而降低恢复复杂度。 受到现有的压缩感知方法的启发，我们提出了无线电深度递归网络（RadioDUN），以展开优化过程，在可学习的方式中实现自适应参数调整和先验拟合。为了考虑无线电传播特性，我们开发了一个动态重加权模块（DRM）来自适应地建模无线电图中每个因子的重要性。受到物理传播模型中的阴影因素的启发，我们将与障碍物相关的因素整合进来以表示由障碍物引起的信号随机衰减。 设计了阴影损失函数进一步约束因子预测，并作为补充监督目标，增强了RadioDUN的表现性能。我们进行了大量的实验来证明所提出的方法优于当前最先进的方法。我们的代码将在发表后公开发布。
URL
https://arxiv.org/abs/2506.08418
PDF
-
Spectrum from Defocus: Fast Spectral Imaging with Chromatic Focal Stack
2025-03-26 03:17:43M. Kerem Aydin, Yi-Chun Hung, Jaclyn Pytlarz, Qi Guo, Emma AlexanderAbstract
Hyperspectral cameras face harsh trade-offs between spatial, spectral, and temporal resolution in an inherently low-photon regime. Computational imaging systems break through these trade-offs with compressive sensing, but require complex optics and/or extensive compute. We present Spectrum from Defocus (SfD), a chromatic focal sweep method that recovers state-of-the-art hyperspectral images with a small system of off-the-shelf optics and < 1 second of compute. Our camera uses two lenses and a grayscale sensor to preserve nearly all incident light in a chromatically-aberrated focal stack. Our physics-based iterative algorithm efficiently demixes, deconvolves, and denoises the blurry grayscale focal stack into a sharp spectral image. The combination of photon efficiency, optical simplicity, and physical modeling makes SfD a promising solution for fast, compact, interpretable hyperspectral imaging.
Abstract (translated)
高光谱相机在低光子环境下面临着空间分辨率、光谱分辨率和时间分辨率之间的严峻权衡。计算成像系统通过压缩感知突破了这些限制，但需要复杂的光学设备和/或大量的计算资源。我们提出了“从脱焦中恢复光谱（SfD）”方法，这是一种色差焦点扫描技术，仅使用一套现成的光学元件及不到1秒的计算时间即可获取最先进的高光谱图像。 我们的相机采用两片透镜和一个灰度传感器来保留几乎所有的入射光线，并将它们存储在一个具有色差的焦点堆栈中。我们基于物理原理的迭代算法可以高效地对模糊的灰度焦点堆栈进行解混、反卷积以及去噪，从而生成清晰的光谱图像。 这种方法在光子效率、光学设计简洁性及物理模型应用方面的结合使其成为快速、紧凑且易于理解的高光谱成像解决方案。
URL
https://arxiv.org/abs/2503.20184
PDF
-
Sampling Innovation-Based Adaptive Compressive Sensing
2025-03-17 14:54:13Zhifu Tian, Tao Hu, Chaoyang Niu, Di Wu, Shu WangAbstract
Scene-aware Adaptive Compressive Sensing (ACS) has attracted significant interest due to its promising capability for efficient and high-fidelity acquisition of scene images. ACS typically prescribes adaptive sampling allocation (ASA) based on previous samples in the absence of ground truth. However, when confronting unknown scenes, existing ACS methods often lack accurate judgment and robust feedback mechanisms for ASA, thus limiting the high-fidelity sensing of the scene. In this paper, we introduce a Sampling Innovation-Based ACS (SIB-ACS) method that can effectively identify and allocate sampling to challenging image reconstruction areas, culminating in high-fidelity image reconstruction. An innovation criterion is proposed to judge ASA by predicting the decrease in image reconstruction error attributable to sampling increments, thereby directing more samples towards regions where the reconstruction error diminishes significantly. A sampling innovation-guided multi-stage adaptive sampling (AS) framework is proposed, which iteratively refines the ASA through a multi-stage feedback process. For image reconstruction, we propose a Principal Component Compressed Domain Network (PCCD-Net), which efficiently and faithfully reconstructs images under AS scenarios. Extensive experiments demonstrate that the proposed SIB-ACS method significantly outperforms the state-of-the-art methods in terms of image reconstruction fidelity and visual effects. Codes are available at this https URL.
Abstract (translated)
场景感知自适应压缩传感（ACS）因其能够高效且高质量地获取场景图像的潜力而引起了广泛的关注。在没有真实情况数据的情况下，ACS通常基于先前样本规定自适应采样分配（ASA）。然而，在面对未知场景时，现有的ACS方法往往难以准确判断并缺乏稳健的反馈机制来改进ASA，从而限制了对场景的高保真度感知。 本文介绍了一种基于采样创新的ACS（SIB-ACS）方法，该方法能够有效识别和优先分配采样到那些具有挑战性的图像重建区域上，最终实现高质量的图像重建。文中提出一个创新标准来判断ASA，通过预测由于增加采样而导致的图像重建误差减少程度，从而将更多的样本集中于重建误差显著降低的区域。 同时提出了基于采样创新引导的多阶段自适应采样（AS）框架，在这个框架下，通过多次迭代反馈过程不断优化ASA。在图像重建方面，我们提出了一种主成分压缩域网络（PCCD-Net），该网络能够高效且准确地在自适应采样的场景中重建图像。 广泛的实验表明，所提出的SIB-ACS方法在图像重建保真度和视觉效果上均显著优于当前最先进的方法。代码可在提供的链接处获取。
URL
https://arxiv.org/abs/2503.13241
PDF
-
Using Powerful Prior Knowledge of Diffusion Model in Deep Unfolding Networks for Image Compressive Sensing
2025-03-11 13:40:46Chen Liao, Yan Shen, Dan Li, Zhongli WangAbstract
Recently, Deep Unfolding Networks (DUNs) have achieved impressive reconstruction quality in the field of image Compressive Sensing (CS) by unfolding iterative optimization algorithms into neural networks. The reconstruction quality of DUNs depends on the learned prior knowledge, so introducing stronger prior knowledge can further improve reconstruction quality. On the other hand, pre-trained diffusion models contain powerful prior knowledge and have a solid theoretical foundation and strong scalability, but it requires a large number of iterative steps to achieve reconstruction. In this paper, we propose to use the powerful prior knowledge of pre-trained diffusion model in DUNs to achieve high-quality reconstruction with less steps for image CS. Specifically, we first design an iterative optimization algorithm named Diffusion Message Passing (DMP), which embeds a pre-trained diffusion model into each iteration process of DMP. Then, we deeply unfold the DMP algorithm into a neural network named DMP-DUN. The proposed DMP-DUN can use lightweight neural networks to achieve mapping from measurement data to the intermediate steps of the reverse diffusion process and directly approximate the divergence of the diffusion model, thereby further improving reconstruction efficiency. Extensive experiments show that our proposed DMP-DUN achieves state-of-the-art performance and requires at least only 2 steps to reconstruct the image. Codes are available at this https URL.
Abstract (translated)
最近，深度展开网络（DUNs）通过将迭代优化算法展开放到神经网络中，在图像压缩感知（CS）领域实现了令人印象深刻的重建质量。DUNs的重建质量依赖于所学习的先验知识，因此引入更强的先验知识可以进一步提高重建质量。另一方面，预训练扩散模型含有强大的先验知识，并具有坚实的理论基础和很强的扩展性，但需要大量的迭代步骤才能实现重建。在本文中，我们提出将预训练扩散模型的强大先验知识用于DUNs，以通过较少的步骤实现高质量的图像CS重建。 具体来说，我们首先设计了一个名为扩散信息传递（DMP）的迭代优化算法，在每次DMP迭代过程中嵌入一个预训练的扩散模型。然后，我们将DMP算法深度展开成一个称为DMP-DUN的神经网络。所提出的DMP-DUN可以使用轻量级的神经网络来实现从测量数据到逆向扩散过程中间步骤的映射，并直接逼近扩散模型的散度，从而进一步提高重建效率。 广泛的实验表明，我们提出的方法DMP-DUN达到了最先进的性能，并且只需要最少2步就能重建图像。代码可在该链接获取：[请在此处插入实际URL]。
URL
https://arxiv.org/abs/2503.08429
PDF
-
Analysis and Synthesis Denoisers for Forward-Backward Plug-and-Play Algorithms
2024-11-20 12:43:40Matthieu Kowalski, Beno\^it Mal\'ezieux, Thomas Moreau, Audrey RepettiAbstract
In this work we study the behavior of the forward-backward (FB) algorithm when the proximity operator is replaced by a sub-iterative procedure to approximate a Gaussian denoiser, in a Plug-and-Play (PnP) fashion. In particular, we consider both analysis and synthesis Gaussian denoisers within a dictionary framework, obtained by unrolling dual-FB iterations or FB iterations, respectively. We analyze the associated minimization problems as well as the asymptotic behavior of the resulting FB-PnP iterations. In particular, we show that the synthesis Gaussian denoising problem can be viewed as a proximity operator. For each case, analysis and synthesis, we show that the FB-PnP algorithms solve the same problem whether we use only one or an infinite number of sub-iteration to solve the denoising problem at each iteration. To this aim, we show that each "one sub-iteration" strategy within the FB-PnP can be interpreted as a primal-dual algorithm when a warm-restart strategy is used. We further present similar results when using a Moreau-Yosida smoothing of the global problem, for an arbitrary number of sub-iterations. Finally, we provide numerical simulations to illustrate our theoretical results. In particular we first consider a toy compressive sensing example, as well as an image restoration problem in a deep dictionary framework.
Abstract (translated)
在这项工作中，我们研究了在插件式（PnP）框架下用子迭代过程近似高斯去噪器时前向后向（FB）算法的行为。特别地，在字典框架内，我们将分析和综合高斯去噪器分别通过展开对偶-FB迭代或FB迭代获得。我们分析了相关的最小化问题以及由此产生的FB-PnP迭代的渐进行为。特别是，我们展示了合成高斯降噪问题可以被视为邻近算子。对于每种情况，无论是分析还是综合，我们都证明了无论是在每次迭代中只使用一次还是无限次子迭代来解决去噪问题，FB-PnP算法都会求解相同的问题。为此，我们展示了在使用热启动策略时，FB-PnP中的“单次子迭代”策略可以被解释为一种原始对偶算法。此外，当使用任意数量的子迭代进行全局问题的Moreau-Yosida平滑处理时，我们也提供了类似的结果。最后，我们通过数值模拟来说明我们的理论结果。特别地，首先考虑了一个玩具压缩感知示例以及深度字典框架下的图像恢复问题。
URL
https://arxiv.org/abs/2411.13276
PDF
-
Block based Adaptive Compressive Sensing with Sampling Rate Control
2024-11-15 13:58:56Kosuke Iwama, Ryugo Morita, Jinjia ZhouAbstract
Compressive sensing (CS), acquiring and reconstructing signals below the Nyquist rate, has great potential in image and video acquisition to exploit data redundancy and greatly reduce the amount of sampled data. To further reduce the sampled data while keeping the video quality, this paper explores the temporal redundancy in video CS and proposes a block based adaptive compressive sensing framework with a sampling rate (SR) control strategy. To avoid redundant compression of non-moving regions, we first incorporate moving block detection between consecutive frames, and only transmit the measurements of moving blocks. The non-moving regions are reconstructed from the previous frame. In addition, we propose a block storage system and a dynamic threshold to achieve adaptive SR allocation to each frame based on the area of moving regions and target SR for controlling the average SR within the target SR. Finally, to reduce blocking artifacts and improve reconstruction quality, we adopt a cooperative reconstruction of the moving and non-moving blocks by referring to the measurements of the non-moving blocks from the previous frame. Extensive experiments have demonstrated that this work is able to control SR and obtain better performance than existing works.
Abstract (translated)
压缩感知（CS），即在低于奈奎斯特率的情况下获取和重建信号，具有极大的潜力通过利用数据冗余来大幅度减少采样数据量，在图像和视频采集方面尤为突出。为了进一步减少采样数据同时保持视频质量，本文探索了视频CS中的时间冗余，并提出了一种基于块的自适应压缩感知框架，以及一种采样率（SR）控制策略。为了避免对非移动区域进行重复压缩，我们首先在连续帧之间加入了移动块检测功能，仅传输移动块的测量值。非移动区域则从前一帧中重建出来。此外，我们提出了一种块存储系统和动态阈值方法，根据移动区域的面积以及目标SR来实现对每帧自适应分配采样率，从而控制整体平均SR维持在目标SR水平内。最后，为了减少阻塞效应并提高重建质量，我们采用了通过参考前一帧中非移动块测量值的合作式移动和非移动区块重建方法。大量的实验表明，这项工作能够有效控制采样率，并且相比现有的研究取得了更好的性能表现。
URL
https://arxiv.org/abs/2411.10200
PDF
-
Chasing Better Deep Image Priors between Over- and Under-parameterization
2024-10-31 17:49:44Qiming Wu, Xiaohan Chen, Yifan Jiang, Zhangyang WangAbstract
Deep Neural Networks (DNNs) are well-known to act as over-parameterized deep image priors (DIP) that regularize various image inverse problems. Meanwhile, researchers also proposed extremely compact, under-parameterized image priors (e.g., deep decoder) that are strikingly competent for image restoration too, despite a loss of accuracy. These two extremes push us to think whether there exists a better solution in the middle: between over- and under-parameterized image priors, can one identify "intermediate" parameterized image priors that achieve better trade-offs between performance, efficiency, and even preserving strong transferability? Drawing inspirations from the lottery ticket hypothesis (LTH), we conjecture and study a novel "lottery image prior" (LIP) by exploiting DNN inherent sparsity, stated as: given an over-parameterized DNN-based image prior, it will contain a sparse subnetwork that can be trained in isolation, to match the original DNN's performance when being applied as a prior to various image inverse problems. Our results validate the superiority of LIPs: we can successfully locate the LIP subnetworks from over-parameterized DIPs at substantial sparsity ranges. Those LIP subnetworks significantly outperform deep decoders under comparably compact model sizes (by often fully preserving the effectiveness of their over-parameterized counterparts), and they also possess high transferability across different images as well as restoration task types. Besides, we also extend LIP to compressive sensing image reconstruction, where a pre-trained GAN generator is used as the prior (in contrast to untrained DIP or deep decoder), and confirm its validity in this setting too. To our best knowledge, this is the first time that LTH is demonstrated to be relevant in the context of inverse problems or image priors.
Abstract (translated)
深度神经网络（DNNs）作为过度参数化的深层图像先验（DIP），已知能够正则化各种图像逆问题。与此同时，研究者也提出了极其紧凑、参数不足的图像先验（例如：深解码器），尽管在准确性上有所损失，但它们对图像恢复同样表现出色。这两种极端促使我们思考是否存在一个更好的中间解决方案：在过度和参数不足的图像先验之间，能否找到实现性能、效率以及保持强迁移能力之间更好平衡的“中间”参数化图像先验？借鉴彩票假设（LTH），我们提出了并研究了一种新颖的“彩票图像先验”（LIP）概念，通过利用DNN内在稀疏性来定义：给定一个基于过度参数化的DNN的图像先验，它将包含一个可以单独训练的稀疏子网络，在应用于各种图像逆问题时可匹配原始DNN的表现。我们的结果验证了LIP的优越性：我们可以在显著的稀疏范围内成功地从过度参数化的DIP中定位到LIP子网络。这些LIP子网络在与深解码器相当紧凑的模型尺寸下表现明显更优（通常完全保留其过度参数化对应物的有效性），并且它们还具备跨不同图像和恢复任务类型的高迁移能力。此外，我们还将LIP扩展到了压缩感知图像重建中，在这里使用预训练的GAN生成器作为先验（与未经过训练的DIP或深解码器相对比），并在这种情况下也证实了其有效性。据我们所知，这是首次在逆问题或图像先验背景下证明LTH的相关性。
URL
https://arxiv.org/abs/2410.24187
PDF
-
A Lightweight Human Pose Estimation Approach for Edge Computing-Enabled Metaverse with Compressive Sensing
2024-08-26 02:57:23Nguyen Quang Hieu, Dinh Thai Hoang, Diep N. NguyenAbstract
The ability to estimate 3D movements of users over edge computing-enabled networks, such as 5G/6G networks, is a key enabler for the new era of extended reality (XR) and Metaverse applications. Recent advancements in deep learning have shown advantages over optimization techniques for estimating 3D human poses given spare measurements from sensor signals, i.e., inertial measurement unit (IMU) sensors attached to the XR devices. However, the existing works lack applicability to wireless systems, where transmitting the IMU signals over noisy wireless networks poses significant challenges. Furthermore, the potential redundancy of the IMU signals has not been considered, resulting in highly redundant transmissions. In this work, we propose a novel approach for redundancy removal and lightweight transmission of IMU signals over noisy wireless environments. Our approach utilizes a random Gaussian matrix to transform the original signal into a lower-dimensional space. By leveraging the compressive sensing theory, we have proved that the designed Gaussian matrix can project the signal into a lower-dimensional space and preserve the Set-Restricted Eigenvalue condition, subject to a power transmission constraint. Furthermore, we develop a deep generative model at the receiver to recover the original IMU signals from noisy compressed data, thus enabling the creation of 3D human body movements at the receiver for XR and Metaverse applications. Simulation results on a real-world IMU dataset show that our framework can achieve highly accurate 3D human poses of the user using only $82\%$ of the measurements from the original signals. This is comparable to an optimization-based approach, i.e., Lasso, but is an order of magnitude faster.
Abstract (translated)
在扩展现实（XR）和元宇宙应用新时代中，通过边缘计算实现网络中用户3D运动估计的能力是一个关键的推动力。最近在深度学习方面的进步表明，在给定传感器信号的剩余测量的情况下，使用惯性测量单元（IMU）传感器附着的XR设备进行3D人体姿势估计的优化技术具有优势。然而，现有的工作缺乏适用于无线系统的适用性，因为在嘈杂的无线网络中传输IMU信号会带来重大挑战。此外，没有考虑到IMU信号的潜在冗余，导致高度冗余的传输。在这项工作中，我们提出了一种新的方法来去除冗余并轻量传输IMU信号在嘈杂无线环境中的方法。我们的方法利用随机高斯矩阵将原始信号转换为低维空间。通过利用压缩感知理论，我们证明了设计的高斯矩阵可以将信号投影到低维空间，并保持有限功率传输约束下的集合剩余范数条件。此外，我们在接收端开发了一个深度生成模型，用于从嘈杂压缩数据中恢复原始IMU信号，从而实现接收端XR和元宇宙应用中创建3D人体运动。在现实世界的IMU数据集上进行模拟结果表明，我们的框架可以使用仅来自原始信号的82%测量实现高度准确的用户3D姿势。这与基于优化的方法（如Lasso）相当，但速度更快。
URL
https://arxiv.org/abs/2409.00087
PDF
-
Towards Real-time Video Compressive Sensing on Mobile Devices
2024-08-14 13:03:31Miao Cao, Lishun Wang, Huan Wang, Guoqing Wang, Xin YuanAbstract
Video Snapshot Compressive Imaging (SCI) uses a low-speed 2D camera to capture high-speed scenes as snapshot compressed measurements, followed by a reconstruction algorithm to retrieve the high-speed video frames. The fast evolving mobile devices and existing high-performance video SCI reconstruction algorithms motivate us to develop mobile reconstruction methods for real-world applications. Yet, it is still challenging to deploy previous reconstruction algorithms on mobile devices due to the complex inference process, let alone real-time mobile reconstruction. To the best of our knowledge, there is no video SCI reconstruction model designed to run on the mobile devices. Towards this end, in this paper, we present an effective approach for video SCI reconstruction, dubbed MobileSCI, which can run at real-time speed on the mobile devices for the first time. Specifically, we first build a U-shaped 2D convolution-based architecture, which is much more efficient and mobile-friendly than previous state-of-the-art reconstruction methods. Besides, an efficient feature mixing block, based on the channel splitting and shuffling mechanisms, is introduced as a novel bottleneck block of our proposed MobileSCI to alleviate the computational burden. Finally, a customized knowledge distillation strategy is utilized to further improve the reconstruction quality. Extensive results on both simulated and real data show that our proposed MobileSCI can achieve superior reconstruction quality with high efficiency on the mobile devices. Particularly, we can reconstruct a 256 X 256 X 8 snapshot compressed measurement with real-time performance (about 35 FPS) on an iPhone 15. Code is available at this https URL.
Abstract (translated)
视频快照压缩成像（SCI）使用一种速度较低的2D相机来捕捉高速场景的快照压缩测量，然后通过重构算法来检索高速视频帧。快速发展的移动设备以及现有的高性能视频SCI复原算法，促使我们为实际应用开发移动复原方法。然而，将以前的重构算法应用于移动设备仍然具有挑战性，尤其是在实时移动复原方面。据我们所知，还没有专门为移动设备设计的视频SCI复原模型。因此，在本文中，我们提出了一个视频SCI复原的有效方法，被称为移动SCI，可以在移动设备上实现实时速度。具体来说，我们首先构建了一个U形2D卷积基架构，这是比以前最先进的复原方法更高效和便携的架构。此外，我们还引入了一个基于通道分割和重排机制的高效特征混合块，作为我们提出的移动SCI的新瓶颈块，减轻计算负担。最后，我们还采用了一种自定义的 Knowledge Distillation 策略来进一步提高复原质量。在模拟数据和真实数据上的广泛结果表明，与移动设备的高效运行相比，我们提出的移动SCI具有卓越的复原质量。特别地，在iPhone 15上，我们可以在实时性能（约35 FPS）下重构256 X 256 X 8的快照压缩测量。代码可在此处下载：https://www.xcode.com/
URL
https://arxiv.org/abs/2408.07530
PDF
-
AI-Based Beam-Level and Cell-Level Mobility Management for High Speed Railway Communications
2024-07-05 08:23:13Wen Li, Wei Chen, Shiyue Wang, Yuanyuan Zhang, Michail Matthaiou, Bo AiAbstract
High-speed railway (HSR) communications are pivotal for ensuring rail safety, operations, maintenance, and delivering passenger information services. The high speed of trains creates rapidly time-varying wireless channels, increases the signaling overhead, and reduces the system throughput, making it difficult to meet the growing and stringent needs of HSR applications. In this article, we explore artificial intelligence (AI)-based beam-level and cell-level mobility management suitable for HSR communications, including the use cases, inputs, outputs, and key performance indicators (KPI)s of AI models. Particularly, in comparison to traditional down-sampling spatial beam measurements, we show that the compressed spatial multi-beam measurements via compressive sensing lead to improved spatial-temporal beam prediction. Moreover, we demonstrate the performance gains of AI-assisted cell handover over traditional mobile handover mechanisms. In addition, we observe that the proposed approaches to reduce the measurement overhead achieve comparable radio link failure performance with the traditional approach that requires all the beam measurements of all cells, while the former methods can save 50% beam measurement overhead.
Abstract (translated)
高速铁路（HSR）通信对于确保铁路安全、运营、维护和提供旅客信息服务至关重要。高速列车产生的快速时间变化的无线信道增加了信号覆盖，降低了系统带宽，使得满足HSR应用不断增长和日益严格的需求变得困难。在本文中，我们探讨了适用于HSR通信的人工智能（AI）基带和细胞级移动管理，包括使用案例、输入、输出和关键绩效指标（KPI）的AI模型的性能。特别地，与传统下采样空间波测量相比，我们证明了通过压缩感知进行压缩空间多波束测量促进了空间-时间波束预测的改善。此外，我们展示了AI辅助细胞 handover 相对于传统移动 handover 机制的性能提升。此外，我们观察到，通过降低测量开销实现的方法在无线电链路故障性能上与传统方法相媲美，而前者可以节省50%的波束测量开销。
URL
https://arxiv.org/abs/2407.04336
PDF
-
Imaging Signal Recovery Using Neural Network Priors Under Uncertain Forward Model Parameters
2024-05-05 14:12:48Xiwen Chen, Wenhui Zhu, Peijie Qiu, Abolfazl RaziAbstract
Inverse imaging problems (IIPs) arise in various applications, with the main objective of reconstructing an image from its compressed measurements. This problem is often ill-posed for being under-determined with multiple interchangeably consistent solutions. The best solution inherently depends on prior knowledge or assumptions, such as the sparsity of the image. Furthermore, the reconstruction process for most IIPs relies significantly on the imaging (i.e. forward model) parameters, which might not be fully known, or the measurement device may undergo calibration drifts. These uncertainties in the forward model create substantial challenges, where inaccurate reconstructions usually happen when the postulated parameters of the forward model do not fully match the actual ones. In this work, we devoted to tackling accurate reconstruction under the context of a set of possible forward model parameters that exist. Here, we propose a novel Moment-Aggregation (MA) framework that is compatible with the popular IIP solution by using a neural network prior. Specifically, our method can reconstruct the signal by considering all candidate parameters of the forward model simultaneously during the update of the neural network. We theoretically demonstrate the convergence of the MA framework, which has a similar complexity with reconstruction under the known forward model parameters. Proof-of-concept experiments demonstrate that the proposed MA achieves performance comparable to the forward model with the known precise parameter in reconstruction across both compressive sensing and phase retrieval applications, with a PSNR gap of 0.17 to 1.94 over various datasets, including MNIST, X-ray, Glas, and MoNuseg. This highlights our method's significant potential in reconstruction under an uncertain forward model.
Abstract (translated)
逆向成像问题（IIPs）在各种应用中出现，主要目的是从压缩测量中重构图像。这个问题通常是不确定的，因为存在多个可交换的一致解。最好的解决方案本质上取决于先验知识或假设，比如图像的稀疏性。此外，大多数IIP的重建过程对成像（即前向模型）参数的依赖性很大程度上，这些参数可能不完全知道，或者测量设备可能经历标定漂移。这些前向模型的不确定性在很大程度上导致了挑战，而当前向模型的假设参数不等于实际值时，通常会出现不准确的重构。在本文中，我们致力于解决在存在一组可能的前向模型参数的背景下实现准确重构的问题。这里，我们提出了一个新颖的Moment-Aggregation（MA）框架，它与流行的IIP解决方案兼容，并使用神经网络先验。具体来说，我们的方法可以在神经网络更新过程中同时考虑所有前向模型候选参数，从而重构信号。我们理论证明了MA框架的收敛，其与已知前向模型参数下的重构具有相似的复杂性。概念性实验证实，所提出的MA在压缩感知和相位恢复应用中，与已知精确参数的前向模型具有可比较的性能，各数据集的PSNR差值在0.17到1.94之间。这突出了在不确定前向模型的背景下进行重构的我们方法的潜在巨大价值。
URL
https://arxiv.org/abs/2405.02944
PDF
-
Study of the gOMP Algorithm for Recovery of Compressed Sensed Hyperspectral Images
2024-01-26 11:20:11Jon Alvarez Justo, Milica OrlandicAbstract
Hyperspectral Imaging (HSI) is used in a wide range of applications such as remote sensing, yet the transmission of the HS images by communication data links becomes challenging due to the large number of spectral bands that the HS images contain together with the limited data bandwidth available in real applications. Compressive Sensing reduces the images by randomly subsampling the spectral bands of each spatial pixel and then it performs the image reconstruction of all the bands using recovery algorithms which impose sparsity in a certain transform domain. Since the image pixels are not strictly sparse, this work studies a data sparsification pre-processing stage prior to compression to ensure the sparsity of the pixels. The sparsified images are compressed $2.5\times$ and then recovered using the Generalized Orthogonal Matching Pursuit algorithm (gOMP) characterized by high accuracy, low computational requirements and fast convergence. The experiments are performed in five conventional hyperspectral images where the effect of different sparsification levels in the quality of the uncompressed as well as the recovered images is studied. It is concluded that the gOMP algorithm reconstructs the hyperspectral images with higher accuracy as well as faster convergence when the pixels are highly sparsified and hence at the expense of reducing the quality of the recovered images with respect to the original images.
Abstract (translated)
超分辨率成像（HSI）在遥感和许多应用领域中都有广泛应用，然而，通过通信数据链传输HS图像变得具有挑战性，因为HS图像包含大量共同光谱带，同时现实应用中可用的数据带宽有限。压缩感知通过随机采样每个空间像素的频率带，然后使用恢复算法对所有频率带进行图像重建，这些算法在某个变换域中引入了稀疏性。由于图像像素不是严格稀疏的，因此本文研究了在压缩之前进行数据稀疏化预处理阶段，以确保像素的稀疏性。稀疏化的图像被压缩2.5倍，然后使用高精度、低计算要求和快速收敛的广义欧几里得匹配 pursuit（gOMP）算法进行恢复。实验在五种常规超分辨率图像上进行，研究了不同稀疏度级别对压缩和恢复图像质量的影响。得出的结论是，当像素高度稀疏时，gOMP算法在压缩和恢复超分辨率图像时具有更高的准确性和更快的收敛速度，但会降低恢复图像与原始图像之间的质量。
URL
https://arxiv.org/abs/2401.14786
PDF
-
A Comparative Study of Compressive Sensing Algorithms for Hyperspectral Imaging Reconstruction
2024-01-26 10:38:39Jon Alvarez Justo, Daniela Lupu, Milica Orlandic, Ion Necoara, Tor Arne JohansenAbstract
Hyperspectral Imaging comprises excessive data consequently leading to significant challenges for data processing, storage and transmission. Compressive Sensing has been used in the field of Hyperspectral Imaging as a technique to compress the large amount of data. This work addresses the recovery of hyperspectral images 2.5x compressed. A comparative study in terms of the accuracy and the performance of the convex FISTA/ADMM in addition to the greedy gOMP/BIHT/CoSaMP recovery algorithms is presented. The results indicate that the algorithms recover successfully the compressed data, yet the gOMP algorithm achieves superior accuracy and faster recovery in comparison to the other algorithms at the expense of high dependence on unknown sparsity level of the data to recover.
Abstract (translated)
超分辨率成像包括 excessive data，因此对数据处理、存储和传输带来显著挑战。在超分辨率成像领域，压缩感知是一种用于压缩大量数据的压缩技术。本文研究了在超分辨率成像中恢复压缩后超分辨率图像2.5倍压缩的情况。在超分辨率图像恢复方面，对凸FISTA/ADMM的准确性和性能进行了比较研究，还包括贪婪梯度OMP/BIHT/CoSaMP恢复算法。结果显示，这些算法成功地恢复压缩后的数据，然而，与其它算法相比，gOMP算法在准确性和恢复速度方面具有优越性，但数据的不确定稀疏水平对算法恢复效果的影响较大。
URL
https://arxiv.org/abs/2401.14762
PDF
-
SnapCap: Efficient Snapshot Compressive Video Captioning
2024-01-10 03:11:21Jianqiao Sun, Yudi Su, Hao Zhang, Ziheng Cheng, Zequn Zeng, Zhengjue Wang, Bo Chen, Xin YuanAbstract
Video Captioning (VC) is a challenging multi-modal task since it requires describing the scene in language by understanding various and complex videos. For machines, the traditional VC follows the "imaging-compression-decoding-and-then-captioning" pipeline, where compression is pivot for storage and transmission. However, in such a pipeline, some potential shortcomings are inevitable, i.e., information redundancy resulting in low efficiency and information loss during the sampling process for captioning. To address these problems, in this paper, we propose a novel VC pipeline to generate captions directly from the compressed measurement, which can be captured by a snapshot compressive sensing camera and we dub our model SnapCap. To be more specific, benefiting from the signal simulation, we have access to obtain abundant measurement-video-annotation data pairs for our model. Besides, to better extract language-related visual representations from the compressed measurement, we propose to distill the knowledge from videos via a pre-trained CLIP with plentiful language-vision associations to guide the learning of our SnapCap. To demonstrate the effectiveness of SnapCap, we conduct experiments on two widely-used VC datasets. Both the qualitative and quantitative results verify the superiority of our pipeline over conventional VC pipelines. In particular, compared to the "caption-after-reconstruction" methods, our SnapCap can run at least 3$\times$ faster, and achieve better caption results.
Abstract (translated)
视频字幕（VC）是一个具有挑战性的多模态任务，因为需要通过理解各种复杂视频来用语言描述场景。对于机器来说，传统的VC沿着“图像压缩-解码-然后编码”的流程进行，压缩是存储和传输的关键。然而，在这样一个流程中，一些潜在的缺陷是无法避免的，即压缩过程中信息冗余导致效率低和信息丢失。为解决这些问题，本文提出了一种新颖的VC管道，可以直接从压缩测量中生成字幕，可以被快照压缩感知相机捕获，我们称之为SnapCap。 具体来说，利用信号仿真，我们获得了为我们的模型提供丰富测量视频注释数据对的能力。此外，为了更好地从压缩测量中提取与语言相关的视觉表示，我们通过预训练的CLIP（带有丰富语言视觉关联）来蒸馏知识，以指导我们的SnapCap的学习。为了证明SnapCap的有效性，我们在两个广泛使用的VC数据集上进行了实验。两个质量和数量结果证实了我们的管道优越于传统VC管道。特别是，与“在重构后进行编码”的方法相比，我们的SnapCap至少可以快3倍，并实现更好的字幕效果。
URL
https://arxiv.org/abs/2401.04903
PDF
-
MsDC-DEQ-Net: Deep Equilibrium Model with Multi-scale Dilated Convolution for Image Compressive Sensing
2024-01-05 16:25:58Youhao Yu, Richard M. DansereauAbstract
Compressive sensing (CS) is a technique that enables the recovery of sparse signals using fewer measurements than traditional sampling methods. To address the computational challenges of CS reconstruction, our objective is to develop an interpretable and concise neural network model for reconstructing natural images using CS. We achieve this by mapping one step of the iterative shrinkage thresholding algorithm (ISTA) to a deep network block, representing one iteration of ISTA. To enhance learning ability and incorporate structural diversity, we integrate aggregated residual transformations (ResNeXt) and squeeze-and-excitation (SE) mechanisms into the ISTA block. This block serves as a deep equilibrium layer, connected to a semi-tensor product network (STP-Net) for convenient sampling and providing an initial reconstruction. The resulting model, called MsDC-DEQ-Net, exhibits competitive performance compared to state-of-the-art network-based methods. It significantly reduces storage requirements compared to deep unrolling methods, using only one iteration block instead of multiple iterations. Unlike deep unrolling models, MsDC-DEQ-Net can be iteratively used, gradually improving reconstruction accuracy while considering computation trade-offs. Additionally, the model benefits from multi-scale dilated convolutions, further enhancing performance.
Abstract (translated)
压缩感知（CS）是一种使用比传统采样方法更少的测量的技术，以恢复稀疏信号。为了应对CS重建的计算挑战，我们的目标是开发一个可解释且紧凑的神经网络模型，用于使用CS对自然图像进行重建。我们通过将迭代收缩阈值算法（ISTA）的迭代一步映射到深度网络块来实现这一目标，表示ISTA的迭代。为了提高学习能力和包含结构多样性，我们将聚合残差变换（ResNeXt）和压缩和激发（SE）机制集成到ISTA模块中。这个模块充当一个深度平衡层，连接到半张量产品网络（STP-Net），用于方便的采样和提供初始重建。所得到的模型，称为MsDC-DEQ-Net，与基于网络的先进方法相比具有竞争性能。它显著减少了存储需求，只需使用一个迭代块而不是多个迭代块。与深展开模型不同，MsDC-DEQ-Net可以迭代使用，在考虑计算开销的同时，逐渐提高重建准确性。此外，模型还利用多尺度扩散卷积，进一步提高性能。
URL
https://arxiv.org/abs/2401.02884
PDF
-
Deep Regularized Compound Gaussian Network for Solving Linear Inverse Problems
2023-11-28 21:53:57Carter Lyons, Raghu G. Raj, Margaret CheneyAbstract
Incorporating prior information into inverse problems, e.g. via maximum-a-posteriori estimation, is an important technique for facilitating robust inverse problem solutions. In this paper, we devise two novel approaches for linear inverse problems that permit problem-specific statistical prior selections within the compound Gaussian (CG) class of distributions. The CG class subsumes many commonly used priors in signal and image reconstruction methods including those of sparsity-based approaches. The first method developed is an iterative algorithm, called generalized compound Gaussian least squares (G-CG-LS), that minimizes a regularized least squares objective function where the regularization enforces a CG prior. G-CG-LS is then unrolled, or unfolded, to furnish our second method, which is a novel deep regularized (DR) neural network, called DR-CG-Net, that learns the prior information. A detailed computational theory on convergence properties of G-CG-LS and thorough numerical experiments for DR-CG-Net are provided. Due to the comprehensive nature of the CG prior, these experiments show that our unrolled DR-CG-Net outperforms competitive prior art methods in tomographic imaging and compressive sensing, especially in challenging low-training scenarios.
Abstract (translated)
将先验信息融入反问题中,例如通过最大后验估计,是促进稳健反问题解的重要技术。在本文中,我们提出了两种新的线性反问题方法,允许在复合高斯(CG)分布中进行问题特定的统计先验选择。CG类包括许多在信号和图像重建方法中常用的先验,包括基于稀疏度的方法。我们开发的第一种方法是一个迭代算法,称为一般化复合高斯最小二乘(G-CG-LS)方法,它最小化一个正则化最小二乘目标函数,其中正则化强制执行CG先验。G-CG-LS然后展开或展开,以提供我们的第二种方法,即名为DR-CG-Net的新型深度正则化(DR)神经网络,它学习先验信息。关于G-CG-LS的收敛性质的详细计算理论和DR-CG-Net的深入数值实验都在本文中提供了。由于CG先验的全局性,这些实验表明,我们的未展开DR-CG-Net在断层成像和压缩感知领域优于竞争先驱技术,尤其是在具有挑战性的低训练场景中。
URL
https://arxiv.org/abs/2311.17248
PDF
-
Experimental Results of Underwater Sound Speed Profile Inversion by Few-shot Multi-task Learning
2023-10-18 04:53:01Wei Huang, Fan Gao, Junting Wang, Hao ZhangAbstract
Underwater Sound Speed Profile (SSP) distribution has great influence on the propagation mode of acoustic signal, thus the fast and accurate estimation of SSP is of great importance in building underwater observation systems. The state-of-the-art SSP inversion methods include frameworks of matched field processing (MFP), compressive sensing (CS), and feedforeward neural networks (FNN), among which the FNN shows better real-time performance while maintain the same level of accuracy. However, the training of FNN needs quite a lot historical SSP samples, which is diffcult to be satisfied in many ocean areas. This situation is called few-shot learning. To tackle this issue, we propose a multi-task learning (MTL) model with partial parameter sharing among different traning tasks. By MTL, common features could be extracted, thus accelerating the learning process on given tasks, and reducing the demand for reference samples, so as to enhance the generalization ability in few-shot learning. To verify the feasibility and effectiveness of MTL, a deep-ocean experiment was held in April 2023 at the South China Sea. Results shows that MTL outperforms the state-of-the-art methods in terms of accuracy for SSP inversion, while inherits the real-time advantage of FNN during the inversion stage.
Abstract (translated)
水下声速剖面（SSP）分布对声波传播模式有很大的影响，因此快速和准确地估计SSP对构建水下观测系统非常重要。最先进的SSP反演方法包括匹配场处理（MFP）框架、压缩感知（CS）和前馈神经网络（FNN）等。其中，FNN在保持相同准确性的同时具有更好的实时性能。然而，为了训练FNN，需要相当多的历史SSP样本，这在许多海洋区域中是难以满足的。这种情况称为欠样本学习。为了解决这个问题，我们提出了一个多任务学习（MTL）模型，其中不同训练任务之间共享部分参数。通过MTL，可以提取共同特征，从而加速在给定任务上的学习过程，并减少对参考样本的需求，从而增强在欠样本学习中的泛化能力。为了验证MTL的可行性和有效性，2023年4月在南海进行了一个深海实验。结果表明，MTL在SSP反演方面的准确性超过了最先进的方法，而在反演阶段，FNN具有实时优势。
URL
https://arxiv.org/abs/2310.11708
PDF
