Compressive_Sensing
-
Sparse Models for Machine Learning
2023-08-26 21:32:13Jianyi LinAbstract
The sparse modeling is an evident manifestation capturing the parsimony principle just described, and sparse models are widespread in statistics, physics, information sciences, neuroscience, computational mathematics, and so on. In statistics the many applications of sparse modeling span regression, classification tasks, graphical model selection, sparse M-estimators and sparse dimensionality reduction. It is also particularly effective in many statistical and machine learning areas where the primary goal is to discover predictive patterns from data which would enhance our understanding and control of underlying physical, biological, and other natural processes, beyond just building accurate outcome black-box predictors. Common examples include selecting biomarkers in biological procedures, finding relevant brain activity locations which are predictive about brain states and processes based on fMRI data, and identifying network bottlenecks best explaining end-to-end performance. Moreover, the research and applications of efficient recovery of high-dimensional sparse signals from a relatively small number of observations, which is the main focus of compressed sensing or compressive sensing, have rapidly grown and became an extremely intense area of study beyond classical signal processing. Likewise interestingly, sparse modeling is directly related to various artificial vision tasks, such as image denoising, segmentation, restoration and superresolution, object or face detection and recognition in visual scenes, and action recognition. In this manuscript, we provide a brief introduction of the basic theory underlying sparse representation and compressive sensing, and then discuss some methods for recovering sparse solutions to optimization problems in effective way, together with some applications of sparse recovery in a machine learning problem known as sparse dictionary learning.
Abstract (translated)
稀疏建模是一种明显的表现,捕捉到我刚才描述的简洁性原则,稀疏模型在统计、物理、信息科学、神经科学、计算数学等领域广泛应用。在统计中,稀疏建模的许多应用涵盖了回归、分类任务、图形模型选择、稀疏高斯估计和稀疏维度减少。它还在许多统计和机器学习领域中特别有效,其主要目标是从数据中发现预测模式,这将增强我们对基础物理、生物和自然过程的理解和控制,超越了仅仅建立准确的黑盒预测器。常见的例子包括在生物学过程中选择生物标记物、基于FMRI数据的 Brain 活动位置找到相关的脑活动区域、并确定网络瓶颈的最佳解释,最有效地解释整体性能。此外,研究和应用从相对少量的观察数据中高效恢复高维稀疏信号的研究和应用,这是压缩感知或压缩感知的主要关注点,已经迅速增长并成为 classical 信号处理之外极为强烈的研究领域。类似地,稀疏建模直接与各种人工视觉任务相关,例如图像去噪、分割、恢复和超分辨率、视觉场景中的物体或面部检测和识别,以及动作识别。在本文中,我们简要介绍了稀疏表示和压缩感知的基础理论,然后讨论了如何有效地恢复优化问题的稀疏解决方案,以及在稀疏字典学习机器学习问题中的稀疏恢复应用。
URL
https://arxiv.org/abs/2308.13960
PDF
-
Compressive Image Scanning Microscope
2023-07-19 08:55:39Ajay Gunalan, Marco Castello, Simonluca Piazza, Shunlei Li, Alberto Diaspro, Leonardo S. Mattos, Paolo BianchiniAbstract
We present a novel approach to implement compressive sensing in laser scanning microscopes (LSM), specifically in image scanning microscopy (ISM), using a single-photon avalanche diode (SPAD) array detector. Our method addresses two significant limitations in applying compressive sensing to LSM: the time to compute the sampling matrix and the quality of reconstructed images. We employ a fixed sampling strategy, skipping alternate rows and columns during data acquisition, which reduces the number of points scanned by a factor of four and eliminates the need to compute different sampling matrices. By exploiting the parallel images generated by the SPAD array, we improve the quality of the reconstructed compressive-ISM images compared to standard compressive confocal LSM images. Our results demonstrate the effectiveness of our approach in producing higher-quality images with reduced data acquisition time and potential benefits in reducing photobleaching.
Abstract (translated)
我们提出了一种新颖的 approach 来实现激光扫描显微镜(LSM)中 compressive sensing 的实现,特别是图像扫描显微镜(ISM)中。我们使用了单光子并发射二极管(SPAD)阵列探测器来实现这一目标。我们的方法解决了在 LSM 中应用 compressive sensing 的两个重要限制:计算采样矩阵所需的时间以及重建图像的质量。我们采用了固定的采样策略,在数据收集过程中跳过交替的行和列,这减少了扫描点的个数,实现了 four 倍的减少,并取代了计算不同采样矩阵的需求。通过利用 SPAD 数组产生的并行图像,我们提高了重建的 compressive-ISM 图像的质量,与标准 compressive confocal LSM 图像相比。我们的结果表明,我们的方法可以在减少数据收集时间的同时提高图像质量,并可能有助于减少 photobleaching 的效果。
URL
https://arxiv.org/abs/2307.09841
PDF
-
Sampling-Priors-Augmented Deep Unfolding Network for Robust Video Compressive Sensing
2023-07-14 12:05:14Yuhao Huang, Gangrong Qu, Youran GeAbstract
Video Compressed Sensing (VCS) aims to reconstruct multiple frames from one single captured measurement, thus achieving high-speed scene recording with a low-frame-rate sensor. Although there have been impressive advances in VCS recently, those state-of-the-art (SOTA) methods also significantly increase model complexity and suffer from poor generality and robustness, which means that those networks need to be retrained to accommodate the new system. Such limitations hinder the real-time imaging and practical deployment of models. In this work, we propose a Sampling-Priors-Augmented Deep Unfolding Network (SPA-DUN) for efficient and robust VCS reconstruction. Under the optimization-inspired deep unfolding framework, a lightweight and efficient U-net is exploited to downsize the model while improving overall performance. Moreover, the prior knowledge from the sampling model is utilized to dynamically modulate the network features to enable single SPA-DUN to handle arbitrary sampling settings, augmenting interpretability and generality. Extensive experiments on both simulation and real datasets demonstrate that SPA-DUN is not only applicable for various sampling settings with one single model but also achieves SOTA performance with incredible efficiency.
Abstract (translated)
视频压缩感知(VCS)的目标是从单个捕获测量中重构多个帧,从而实现低帧率传感器的高速度场景录制。尽管最近在VCS方面取得了令人印象深刻的进展,但这些先进的方法也显著增加了模型的复杂性并出现了 poor generality和Robustness 的问题,这意味着这些网络需要适应新的系统并进行训练。这些限制妨碍了实时成像和模型的实际部署。在本文中,我们提出了一种采样先验增强深度展开网络(SPA-DUN)来高效和稳健地 VCS 重构。在基于优化的深度展开框架下,利用轻量级且高效的 U-net 减小模型大小并提高整体性能。此外,从采样模型的先验知识动态地调节网络特征,使单个 SPA-DUN 能够处理任意采样设置,增加可解释性和一般性。在模拟和真实数据集上的广泛实验表明,SPA-DUN不仅可以适用于单个模型的各种采样设置,而且具有惊人的效率和 SOTA 性能。
URL
https://arxiv.org/abs/2307.07291
PDF
-
Operational Support Estimator Networks
2023-07-12 10:29:40Mete Ahishali, Mehmet Yamac, Serkan Kiranyaz, Moncef GabboujAbstract
In this work, we propose a novel approach called Operational Support Estimator Networks (OSENs) for the support estimation task. Support Estimation (SE) is defined as finding the locations of non-zero elements in a sparse signal. By its very nature, the mapping between the measurement and sparse signal is a non-linear operation. Traditional support estimators rely on computationally expensive iterative signal recovery techniques to achieve such non-linearity. Contrary to the convolution layers, the proposed OSEN approach consists of operational layers that can learn such complex non-linearities without the need for deep networks. In this way, the performance of the non-iterative support estimation is greatly improved. Moreover, the operational layers comprise so-called generative \textit{super neurons} with non-local kernels. The kernel location for each neuron/feature map is optimized jointly for the SE task during the training. We evaluate the OSENs in three different applications: i. support estimation from Compressive Sensing (CS) measurements, ii. representation-based classification, and iii. learning-aided CS reconstruction where the output of OSENs is used as prior knowledge to the CS algorithm for an enhanced reconstruction. Experimental results show that the proposed approach achieves computational efficiency and outperforms competing methods, especially at low measurement rates by a significant margin. The software implementation is publicly shared at this https URL.
Abstract (translated)
在本作品中,我们提出了一种名为操作支持估计器网络(osen)的新方法,用于支持估计任务。支持估计(SE)的定义是找到稀疏信号中的非零元素的位置。从特性的角度来看,测量和稀疏信号之间的映射是一种非线性操作。传统的支持估计方法依赖于计算代价高昂的迭代信号恢复技术来实现这种非线性。与卷积层相反,我们提出的osen方法包括操作层,这些操作层不需要深度网络就可以学习这些复杂的非线性特性。通过这种方式,非迭代的支持估计性能得到了极大的改善。此外,操作层由所谓的生成神经元(super neurons)组成的,这些神经元具有非局部Kernel。每个神经元/特征映射的Kernel位置在训练期间 jointly 用于支持估计任务。我们在不同的应用程序中评估了osen:i.从压缩感知测量(CS)测量中进行支持估计,ii.基于表示的分类,以及iii.学习辅助的CS重构,其中osen的输出被用作CS算法的前置知识,以增强重构。实验结果表明,我们提出的方法实现了计算效率,并比竞争方法更有效,特别是在低测量率方面具有显著优势。软件实现在此httpsURL上公开分享。
URL
https://arxiv.org/abs/2307.06065
PDF
-
Dynamic Path-Controllable Deep Unfolding Network for Compressive Sensing
2023-06-28 09:49:15Jiechong Song, Bin Chen, Jian ZhangAbstract
Deep unfolding network (DUN) that unfolds the optimization algorithm into a deep neural network has achieved great success in compressive sensing (CS) due to its good interpretability and high performance. Each stage in DUN corresponds to one iteration in optimization. At the test time, all the sampling images generally need to be processed by all stages, which comes at a price of computation burden and is also unnecessary for the images whose contents are easier to restore. In this paper, we focus on CS reconstruction and propose a novel Dynamic Path-Controllable Deep Unfolding Network (DPC-DUN). DPC-DUN with our designed path-controllable selector can dynamically select a rapid and appropriate route for each image and is slimmable by regulating different performance-complexity tradeoffs. Extensive experiments show that our DPC-DUN is highly flexible and can provide excellent performance and dynamic adjustment to get a suitable tradeoff, thus addressing the main requirements to become appealing in practice. Codes are available at this https URL.
Abstract (translated)
展开优化算法并将其展开成深度神经网络的 Deep Un unfold Network (DUN) 在压缩感知(CS)方面取得了巨大的成功,因为其良好的解释性和高性能。DUN 的每一阶段对应着优化中的迭代。在测试时,通常需要对所有采样图像进行处理,这需要计算负担,而对于更容易恢复其内容的图像则没有必要。在本文中,我们专注于 CS 重建并提出了一种新型的动态路径控制 Deep Un unfold Network (DPC-DUN)。DPC-DUN 使用我们设计的可控制路径选择器可以动态地选择每个图像的迅速且适当的路径,并通过调节不同的性能-复杂性权衡来减小规模。广泛的实验表明,我们的 DPC-DUN 非常灵活,可以提供出色的性能和动态调整,以获得适当的权衡,从而满足了在实践中变得吸引人的主要要求。代码可在 this https URL 上获取。
URL
https://arxiv.org/abs/2306.16060
PDF
-
MOSAIC: Masked Optimisation with Selective Attention for Image Reconstruction
2023-06-01 17:05:02Pamuditha Somarathne, Tharindu Wickremasinghe, Amashi Niwarthana, A. Thieshanthan, Chamira U.S. Edussooriya, Dushan N. WadduwageAbstract
Compressive sensing (CS) reconstructs images from sub-Nyquist measurements by solving a sparsity-regularized inverse problem. Traditional CS solvers use iterative optimizers with hand crafted sparsifiers, while early data-driven methods directly learn an inverse mapping from the low-dimensional measurement space to the original image space. The latter outperforms the former, but is restrictive to a pre-defined measurement domain. More recent, deep unrolling methods combine traditional proximal gradient methods and data-driven approaches to iteratively refine an image approximation. To achieve higher accuracy, it has also been suggested to learn both the sampling matrix, and the choice of measurement vectors adaptively. Contrary to the current trend, in this work we hypothesize that a general inverse mapping from a random set of compressed measurements to the image domain exists for a given measurement basis, and can be learned. Such a model is single-shot, non-restrictive and does not parametrize the sampling process. To this end, we propose MOSAIC, a novel compressive sensing framework to reconstruct images given any random selection of measurements, sampled using a fixed basis. Motivated by the uneven distribution of information across measurements, MOSAIC incorporates an embedding technique to efficiently apply attention mechanisms on an encoded sequence of measurements, while dispensing the need to use unrolled deep networks. A range of experiments validate our proposed architecture as a promising alternative for existing CS reconstruction methods, by achieving the state-of-the-art for metrics of reconstruction accuracy on standard datasets.
Abstract (translated)
压缩感知(CS)通过解决稀疏性限制的逆问题，从低 Nyquist 测量值中恢复图像。传统的 CS 解决方法使用迭代优化工具和手工制作的稀疏化器，而早期的数据驱动方法直接学习从低维度测量空间到原始图像空间的逆映射。前者比后者表现更好，但只适用于预定义测量域。更近期，深度展开方法结合了传统的近邻梯度方法和数据驱动方法，以迭代地 refine 图像近似。为了获得更高的精度，也建议自适应地学习采样矩阵和测量向量的选择。与当前趋势相反，在这个研究中，我们假设从一个随机的压缩测量集合到图像域的通用逆映射存在，并且可以学习。这样的模型是一个一次性的，不限制的，并且不需要对采样过程参数化。为此，我们提出了 MOSAIC，一个新型的 CS 恢复框架，使用一个固定的基采样，通过随机选择测量集合来恢复图像。因为测量信息不均衡分布，MOSAIC 采用了嵌入技术，高效应用注意力机制，在编码序列的测量集合上，而不需要展开深层网络。一系列实验验证我们提出的架构作为现有 CS 恢复方法的有前途的替代方案，通过在标准数据集上实现恢复精度的顶级指标。
URL
https://arxiv.org/abs/2306.00906
PDF
-
A Compound Gaussian Network for Solving Linear Inverse Problems
2023-05-18 17:05:09Carter Lyons, Raghu G. Raj, Margaret CheneyAbstract
For solving linear inverse problems, particularly of the type that appear in tomographic imaging and compressive sensing, this paper develops two new approaches. The first approach is an iterative algorithm that minimizers a regularized least squares objective function where the regularization is based on a compound Gaussian prior distribution. The Compound Gaussian prior subsumes many of the commonly used priors in image reconstruction, including those of sparsity-based approaches. The developed iterative algorithm gives rise to the paper's second new approach, which is a deep neural network that corresponds to an "unrolling" or "unfolding" of the iterative algorithm. Unrolled deep neural networks have interpretable layers and outperform standard deep learning methods. This paper includes a detailed computational theory that provides insight into the construction and performance of both algorithms. The conclusion is that both algorithms outperform other state-of-the-art approaches to tomographic image formation and compressive sensing, especially in the difficult regime of low training.
Abstract (translated)
为了解决线性逆问题，特别是出现在磁共振成像和压缩感知中的问题，本文开发了两种新的算法。第一种方法是迭代算法，其最小化的目标是 regularized 最小二乘法 objective function，其中Regularization是基于组合高斯先验分布的。组合高斯先验分布将许多在图像重建中常用的先验包括在内，包括基于密度的先验。开发迭代算法导致本文提出的第二种新算法，这是一种深度神经网络，与迭代算法的“展开”或“展开”对应。展开的深度神经网络具有可解释的层，并比标准深度学习方法表现更好。本文包括详细的计算理论，提供了对两个算法构造和性能的理解。结论是，两个算法在磁共振成像和压缩感知中的表现优于其他先进的方法，特别是在低训练状态下。
URL
https://arxiv.org/abs/2305.11120
PDF
-
Recursions Are All You Need: Towards Efficient Deep Unfolding Networks
2023-05-09 14:54:41Rawwad Alhejaili (1 and 2 and 3), Motaz Alfarraj (1 and 2 and 3), Hamzah Luqman (1 and 4), Ali Al-Shaikhi (1 and 2) ((1) King Fahd University of Petroleum and Minerals, (2) Electrical Engineering Department, (3) SDAIA-KFUPM Joint Research Center for Artificial Intelligence, (4) Information and Computer Science Department)Abstract
The use of deep unfolding networks in compressive sensing (CS) has seen wide success as they provide both simplicity and interpretability. However, since most deep unfolding networks are iterative, this incurs significant redundancies in the network. In this work, we propose a novel recursion-based framework to enhance the efficiency of deep unfolding models. First, recursions are used to effectively eliminate the redundancies in deep unfolding networks. Secondly, we randomize the number of recursions during training to decrease the overall training time. Finally, to effectively utilize the power of recursions, we introduce a learnable unit to modulate the features of the model based on both the total number of iterations and the current iteration index. To evaluate the proposed framework, we apply it to both ISTA-Net+ and COAST. Extensive testing shows that our proposed framework allows the network to cut down as much as 75% of its learnable parameters while mostly maintaining its performance, and at the same time, it cuts around 21% and 42% from the training time for ISTA-Net+ and COAST respectively. Moreover, when presented with a limited training dataset, the recursive models match or even outperform their respective non-recursive baseline. Codes and pretrained models are available at this https URL .
Abstract (translated)
深度展开网络在压缩感知(CS)中广泛应用，因为它们既简单又易于解释。然而，由于大多数深度展开网络是迭代的，因此在网络中造成了巨大的冗余。在本研究中，我们提出了一种基于递归的新框架，以增强深度展开模型的效率。我们首先利用递归有效地消除深度展开网络中的冗余。其次，我们在训练期间随机化递归次数，以减少整个训练时间。最后，为了有效地利用递归的力量，我们引入一个可学习单元，根据迭代次数的总数量和当前迭代指数来调节模型的特征。为了评估所提出的框架，我们将其应用于ISTA-Net+和COAST。广泛的测试表明，我们提出的框架可以使网络最大限度地减少其可学习参数中的冗余，而大部分时间仍然保持其表现，同时，它分别减少了ISTA-Net+和COAST的训练时间中的约21%和42%。此外，当面对有限的训练数据时，递归模型几乎与或甚至超越了其非递归基线。代码和预训练模型可在本网站上https://url.com获得。
URL
https://arxiv.org/abs/2305.05505
PDF
-
NL-CS Net: Deep Learning with Non-Local Prior for Image Compressive Sensing
2023-05-06 02:34:28Shuai Bian, Shouliang Qi, Chen Li, Yudong Yao, Yueyang TengAbstract
Deep learning has been applied to compressive sensing (CS) of images successfully in recent years. However, existing network-based methods are often trained as the black box, in which the lack of prior knowledge is often the bottleneck for further performance improvement. To overcome this drawback, this paper proposes a novel CS method using non-local prior which combines the interpretability of the traditional optimization methods with the speed of network-based methods, called NL-CS Net. We unroll each phase from iteration of the augmented Lagrangian method solving non-local and sparse regularized optimization problem by a network. NL-CS Net is composed of the up-sampling module and the recovery module. In the up-sampling module, we use learnable up-sampling matrix instead of a predefined one. In the recovery module, patch-wise non-local network is employed to capture long-range feature correspondences. Important parameters involved (e.g. sampling matrix, nonlinear transforms, shrinkage thresholds, step size, $etc.$) are learned end-to-end, rather than hand-crafted. Furthermore, to facilitate practical implementation, orthogonal and binary constraints on the sampling matrix are simultaneously adopted. Extensive experiments on natural images and magnetic resonance imaging (MRI) demonstrate that the proposed method outperforms the state-of-the-art methods while maintaining great interpretability and speed.
Abstract (translated)
深度学习近年来成功地应用于图像压缩感知(CS)中。然而，现有的网络方法往往被训练为黑盒，缺乏先前知识往往成为进一步性能改进的瓶颈。为了克服这一缺点，本文提出了一种使用非局部先前的新型CS方法，该方法将传统的优化方法的解释性与网络方法的速度相结合，称为NL-CSNet。我们从扩展拉格朗日方法的迭代中展开每个阶段，以解决非局部和稀疏 Regularized 优化问题，通过网络实现。NL-CSNet由采样模块和恢复模块组成。在采样模块中，我们使用可学习采样矩阵而不是预先定义的矩阵。在恢复模块中，采用点 wise的非局部网络来捕捉长距离特征对应关系。参与重要参数(例如采样矩阵、非线性变换、收缩阈值、步长大小等)的学习是端到端学习的，而不是手工构建的。此外，为了促进实际实现，同时采用Orthogonal 和二进制采样矩阵约束。对自然图像和磁共振成像(MRI)进行了广泛的实验，证明了该方法在保持极大的解释性和速度优势的同时，击败了最先进的方法。
URL
https://arxiv.org/abs/2305.03899
PDF
-
Optimization-Inspired Cross-Attention Transformer for Compressive Sensing
2023-04-27 07:21:30Jiechong Song, Chong Mou, Shiqi Wang, Siwei Ma, Jian ZhangAbstract
By integrating certain optimization solvers with deep neural networks, deep unfolding network (DUN) with good interpretability and high performance has attracted growing attention in compressive sensing (CS). However, existing DUNs often improve the visual quality at the price of a large number of parameters and have the problem of feature information loss during iteration. In this paper, we propose an Optimization-inspired Cross-attention Transformer (OCT) module as an iterative process, leading to a lightweight OCT-based Unfolding Framework (OCTUF) for image CS. Specifically, we design a novel Dual Cross Attention (Dual-CA) sub-module, which consists of an Inertia-Supplied Cross Attention (ISCA) block and a Projection-Guided Cross Attention (PGCA) block. ISCA block introduces multi-channel inertia forces and increases the memory effect by a cross attention mechanism between adjacent iterations. And, PGCA block achieves an enhanced information interaction, which introduces the inertia force into the gradient descent step through a cross attention block. Extensive CS experiments manifest that our OCTUF achieves superior performance compared to state-of-the-art methods while training lower complexity. Codes are available at this https URL.
Abstract (translated)
通过将某些优化求解器和深度神经网络集成起来，具有良好解释性和高性能的深度展开网络(DUN)在压缩感知(CS)中越来越受到关注。然而，现有的DUN往往通过大量参数来提高视觉质量，并且在迭代过程中会出现特征信息丢失的问题。在本文中，我们提出一种基于优化的交叉注意力Transformer(OCT)模块作为迭代过程，从而生成轻量级的基于OCT的图像展开框架(OCTUF)。具体来说，我们设计了一个独特的双重交叉注意力(Dual-CA)子模块，其中包含一个惯性提供交叉注意力(ISCA)块和一个投影引导交叉注意力(PGCA)块。ISCA块引入了多通道惯性力，并通过相邻迭代中的交叉注意力机制增加记忆效应。PGCA块实现了增强的信息交互，通过交叉注意力块将惯性力引入梯度下降步骤。广泛的CS实验表明，我们的OCTUF在训练复杂性较低的情况下比现有方法表现出更好的性能。代码可在该httpsURL上获取。
URL
https://arxiv.org/abs/2304.13986
PDF
-
Hierarchical Interactive Reconstruction Network For Video Compressive Sensing
2023-04-15 04:57:57Tong Zhang, Wenxue Cui, Chen Hui, Feng JiangAbstract
Deep network-based image and video Compressive Sensing(CS) has attracted increasing attentions in recent years. However, in the existing deep network-based CS methods, a simple stacked convolutional network is usually adopted, which not only weakens the perception of rich contextual prior knowledge, but also limits the exploration of the correlations between temporal video frames. In this paper, we propose a novel Hierarchical InTeractive Video CS Reconstruction Network(HIT-VCSNet), which can cooperatively exploit the deep priors in both spatial and temporal domains to improve the reconstruction quality. Specifically, in the spatial domain, a novel hierarchical structure is designed, which can hierarchically extract deep features from keyframes and non-keyframes. In the temporal domain, a novel hierarchical interaction mechanism is proposed, which can cooperatively learn the correlations among different frames in the multiscale space. Extensive experiments manifest that the proposed HIT-VCSNet outperforms the existing state-of-the-art video and image CS methods in a large margin.
Abstract (translated)
深度学习图像和视频压缩感知(CS)近年来日益受到关注。然而，在现有的深度学习CS方法中，通常采用简单的叠加卷积神经网络，这不仅削弱了丰富的上下文先验知识的感受度，而且也限制了对时间帧之间相关性的探索。在本文中，我们提出了一种新的Hierarchical InTeractive Video CS Reconstruction Network(HIT-VCSNet)，它可以合作利用空间和时间域中的深层先验知识，以提高重建质量。具体来说，在空间域中，我们设计了一种 novel Hierarchical 结构，可以从关键帧和非关键帧中Hierarchically 提取深层特征。在时间域中，我们提出了一种 novel Hierarchical 交互机制，可以在多尺度空间中合作学习不同帧之间的相关性。广泛的实验表明，提出的HIT-VCSNet在显著优于现有的先进的视频和图像CS方法。
URL
https://arxiv.org/abs/2304.07473
PDF
-
Accelerated parallel MRI using memory efficient and robust monotone operator learning
2023-04-03 20:26:59Aniket Pramanik, Mathews JacobAbstract
Model-based deep learning methods that combine imaging physics with learned regularization priors have been emerging as powerful tools for parallel MRI acceleration. The main focus of this paper is to determine the utility of the monotone operator learning (MOL) framework in the parallel MRI setting. The MOL algorithm alternates between a gradient descent step using a monotone convolutional neural network (CNN) and a conjugate gradient algorithm to encourage data consistency. The benefits of this approach include similar guarantees as compressive sensing algorithms including uniqueness, convergence, and stability, while being significantly more memory efficient than unrolled methods. We validate the proposed scheme by comparing it with different unrolled algorithms in the context of accelerated parallel MRI for static and dynamic settings.
Abstract (translated)
基于模型的深度学习方法，将图像物理学与学习后的正则化预处理相结合，正在成为并行MRI加速的强大工具。本文的主要焦点是确定Monotone Operator Learning (MOL)框架在并行MRI设置中的实用性。MOL算法在采用Monotone卷积神经网络(CNN)和Conjugate Gradient算法的梯度下降步骤之间交替进行，以鼓励数据一致性。这种方法的好处包括与压缩感知算法类似的保证，包括独特性、收敛和稳定性，而比展开方法更有效地利用了内存。我们验证了所提出的 scheme 的方法，并将其与不同的展开算法在加速静态和动态MRI设置中的并行加速比较。
URL
https://arxiv.org/abs/2304.01351
PDF
-
Compressive Sensing with Tensorized Autoencoder
2023-03-10 22:59:09Rakib Hyder, M. Salman AsifAbstract
Deep networks can be trained to map images into a low-dimensional latent space. In many cases, different images in a collection are articulated versions of one another; for example, same object with different lighting, background, or pose. Furthermore, in many cases, parts of images can be corrupted by noise or missing entries. In this paper, our goal is to recover images without access to the ground-truth (clean) images using the articulations as structural prior of the data. Such recovery problems fall under the domain of compressive sensing. We propose to learn autoencoder with tensor ring factorization on the the embedding space to impose structural constraints on the data. In particular, we use a tensor ring structure in the bottleneck layer of the autoencoder that utilizes the soft labels of the structured dataset. We empirically demonstrate the effectiveness of the proposed approach for inpainting and denoising applications. The resulting method achieves better reconstruction quality compared to other generative prior-based self-supervised recovery approaches for compressive sensing.
Abstract (translated)
深度网络可以被训练将图像映射到低维度的潜在空间。在许多情况下,一组图像是相互联系的版本;例如,相同的物体以不同的照明、背景或姿态呈现。此外,在许多情况下,图像的部分可以受到噪声或缺失值的影响。在本文中,我们的的目标是使用连接作为数据的结构先验来恢复图像,而连接则作为数据的结构约束。这种恢复问题属于压缩感知技术的范围。我们提议学习使用 Tensor环乘法将嵌入空间中的 Tensor环表示作为编码器的结构先验,并施加数据的结构约束。特别,我们将在编码器的瓶颈层中使用 Tensor环结构,利用结构数据集的软标签。我们经验证了该方法在填充和去噪应用中的的有效性。结果方法相对于其他基于生成先验的自监督恢复方法来说,实现了更好的重建质量。
URL
https://arxiv.org/abs/2303.06235
PDF
-
Multilinear compressive sensing and an application to convolutional linear networks
2023-02-02 07:37:58François Malgouyres (IMT), Joseph Landsberg (TAMU)Abstract
We study a deep linear network endowed with a structure. It takes the form of a matrix $X$ obtained by multiplying $K$ matrices (called factors and corresponding to the action of the layers). The action of each layer (i.e. a factor) is obtained by applying a fixed linear operator to a vector of parameters satisfying a constraint. The number of layers is not limited. Assuming that $X$ is given and factors have been estimated, the error between the product of the estimated factors and $X$ (i.e. the reconstruction error) is either the statistical or the empirical risk. In this paper, we provide necessary and sufficient conditions on the network topology under which a stability property holds. The stability property requires that the error on the parameters defining the factors (i.e. the stability of the recovered parameters) scales linearly with the reconstruction error (i.e. the risk). Therefore, under these conditions on the network topology, any successful learning task leads to stably defined features and therefore interpretable layers/network.In order to do so, we first evaluate how the Segre embedding and its inverse distort distances. Then, we show that any deep structured linear network can be cast as a generic multilinear problem (that uses the Segre embedding). This is the {\em tensorial lifting}. Using the tensorial lifting, we provide necessary and sufficient conditions for the identifiability of the factors (up to a scale rearrangement). We finally provide the necessary and sufficient condition called \NSPlong~(because of the analogy with the usual Null Space Property in the compressed sensing framework) which guarantees that the stability property holds. We illustrate the theory with a practical example where the deep structured linear network is a convolutional linear network. As expected, the conditions are rather strong but not empty. A simple test on the network topology can be implemented to test if the condition holds.
Abstract (translated)
我们研究了一个具有结构的深度学习线性网络。它的形式是 $K$ 个矩阵的乘积得到的矩阵 $X$,即称为 factors 并对应每个层的动作。每个层的动作(即一个 factors)是通过应用一个固定的线性操作对满足约束的参数向量进行计算得到的。层的数量不受限制。假设 $X$ 已经给定,并已经估计了 factors,则估计的 factors 与 $X$ 的乘积之差(即重建误差)可能是统计风险或经验风险。在本文中,我们提供了在网络拓扑下保持稳定性的必要条件和充分条件。稳定性性质要求定义 factors 的参数误差(即恢复参数的稳定性)以线性方式与重建误差(即风险)成正比。因此,在这些网络拓扑下,任何成功的学习任务都会导致稳定定义的特征,因此可解释的层/网络。为了这样做,我们首先评估了史格雷嵌入和它的逆对距离的影响。然后,我们表明,任何具有深刻结构深度学习线性网络都可以转换为一个通用多线性问题(使用史格雷嵌入)。这就是 Tensorial lifting。使用 Tensorial lifting,我们提供了 factors 的可确定性的必要条件和充分条件(到规模重构)。最后,我们提供了称为 NSPlong~(因为与压缩感知框架中的常见 null 空间 property 的类比)的必要和充分条件,以确保稳定性性质成立。我们使用实际示例来阐明理论,其中深刻的结构深度学习线性网络是一个卷积线性网络。正如预期的那样,条件相当强,但不是空的。可以通过网络拓扑的简单测试来测试条件是否成立。
URL
https://arxiv.org/abs/1703.08044
PDF
-
LR-CSNet: Low-Rank Deep Unfolding Network for Image Compressive Sensing
2022-12-18 13:54:11Tianfang Zhang, Lei Li, Christian Igel, Stefan Oehmcke, Fabian Gieseke, Zhenming PengAbstract
Deep unfolding networks (DUNs) have proven to be a viable approach to compressive sensing (CS). In this work, we propose a DUN called low-rank CS network (LR-CSNet) for natural image CS. Real-world image patches are often well-represented by low-rank approximations. LR-CSNet exploits this property by adding a low-rank prior to the CS optimization task. We derive a corresponding iterative optimization procedure using variable splitting, which is then translated to a new DUN architecture. The architecture uses low-rank generation modules (LRGMs), which learn low-rank matrix factorizations, as well as gradient descent and proximal mappings (GDPMs), which are proposed to extract high-frequency features to refine image details. In addition, the deep features generated at each reconstruction stage in the DUN are transferred between stages to boost the performance. Our extensive experiments on three widely considered datasets demonstrate the promising performance of LR-CSNet compared to state-of-the-art methods in natural image CS.
Abstract (translated)
URL
https://arxiv.org/abs/2212.09088
PDF
-
A Model-data-driven Network Embedding Multidimensional Features for Tomographic SAR Imaging
2022-11-28 02:01:43Yu Ren, Xiaoling Zhang, Xu Zhan, Jun Shi, Shunjun Wei, Tianjiao ZengAbstract
Deep learning (DL)-based tomographic SAR imaging algorithms are gradually being studied. Typically, they use an unfolding network to mimic the iterative calculation of the classical compressive sensing (CS)-based methods and process each range-azimuth unit individually. However, only one-dimensional features are effectively utilized in this way. The correlation between adjacent resolution units is ignored directly. To address that, we propose a new model-data-driven network to achieve tomoSAR imaging based on multi-dimensional features. Guided by the deep unfolding methodology, a two-dimensional deep unfolding imaging network is constructed. On the basis of it, we add two 2D processing modules, both convolutional encoder-decoder structures, to enhance multi-dimensional features of the imaging scene effectively. Meanwhile, to train the proposed multifeature-based imaging network, we construct a tomoSAR simulation dataset consisting entirely of simulation data of buildings. Experiments verify the effectiveness of the model. Compared with the conventional CS-based FISTA method and DL-based gamma-Net method, the result of our proposed method has better performance on completeness while having decent imaging accuracy.
Abstract (translated)
URL
https://arxiv.org/abs/2211.15002
PDF
-
JSRNN: Joint Sampling and Reconstruction Neural Networks for High Quality Image Compressed Sensing
2022-11-11 02:20:30Chunyan Zeng, Jiaxiang Ye, Zhifeng Wang, Nan Zhao, Minghu WuAbstract
Most Deep Learning (DL) based Compressed Sensing (DCS) algorithms adopt a single neural network for signal reconstruction, and fail to jointly consider the influences of the sampling operation for reconstruction. In this paper, we propose unified framework, which jointly considers the sampling and reconstruction process for image compressive sensing based on well-designed cascade neural networks. Two sub-networks, which are the sampling sub-network and the reconstruction sub-network, are included in the proposed framework. In the sampling sub-network, an adaptive full connected layer instead of the traditional random matrix is used to mimic the sampling operator. In the reconstruction sub-network, a cascade network combining stacked denoising autoencoder (SDA) and convolutional neural network (CNN) is designed to reconstruct signals. The SDA is used to solve the signal mapping problem and the signals are initially reconstructed. Furthermore, CNN is used to fully recover the structure and texture features of the image to obtain better reconstruction performance. Extensive experiments show that this framework outperforms many other state-of-the-art methods, especially at low sampling rates.
Abstract (translated)
URL
https://arxiv.org/abs/2211.05963
PDF
-
Fast Disparity Estimation from a Single Compressed Light Field Measurement
2022-09-22 22:59:09Emmanuel Martinez, Edwin Vargas, Henry ArguelloAbstract
The abundant spatial and angular information from light fields has allowed the development of multiple disparity estimation approaches. However, the acquisition of light fields requires high storage and processing cost, limiting the use of this technology in practical applications. To overcome these drawbacks, the compressive sensing (CS) theory has allowed the development of optical architectures to acquire a single coded light field measurement. This measurement is decoded using an optimization algorithm or deep neural network that requires high computational costs. The traditional approach for disparity estimation from compressed light fields requires first recovering the entire light field and then a post-processing step, thus requiring long times. In contrast, this work proposes a fast disparity estimation from a single compressed measurement by omitting the recovery step required in traditional approaches. Specifically, we propose to jointly optimize an optical architecture for acquiring a single coded light field snapshot and a convolutional neural network (CNN) for estimating the disparity maps. Experimentally, the proposed method estimates disparity maps comparable with those obtained from light fields reconstructed using deep learning approaches. Furthermore, the proposed method is 20 times faster in training and inference than the best method that estimates the disparity from reconstructed light fields.
Abstract (translated)
URL
https://arxiv.org/abs/2209.11342
PDF
-
Uformer-ICS: A Specialized U-Shaped Transformer for Image Compressive Sensing
2022-09-05 04:52:12Kuiyuan Zhang, Zhongyun Hua, Yuanman Li, Yushu Zhang, Yicong ZhouAbstract
Recently, several studies have applied deep convolutional neural networks (CNNs) in image compressive sensing (CS) tasks to improve reconstruction quality. However, convolutional layers generally have a small receptive field; therefore, capturing long-range pixel correlations using CNNs is challenging, which limits their reconstruction performance in image CS tasks. Considering this limitation, we propose a U-shaped transformer for image CS tasks, called the Uformer-ICS. We develop a projection-based transformer block by integrating the prior projection knowledge of CS into the original transformer blocks, and then build a symmetrical reconstruction model using the projection-based transformer blocks and residual convolutional blocks. Compared with previous CNN-based CS methods that can only exploit local image features, the proposed reconstruction model can simultaneously utilize the local features and long-range dependencies of an image, and the prior projection knowledge of the CS theory. Additionally, we design an adaptive sampling model that can adaptively sample image blocks based on block sparsity, which can ensure that the compressed results retain the maximum possible information of the original image under a fixed sampling ratio. The proposed Uformer-ICS is an end-to-end framework that simultaneously learns the sampling and reconstruction processes. Experimental results demonstrate that it achieves significantly better reconstruction performance than existing state-of-the-art deep learning-based CS methods.
Abstract (translated)
URL
https://arxiv.org/abs/2209.01763
PDF
-
D3C2-Net: Dual-Domain Deep Convolutional Coding Network for Compressive Sensing
2022-07-27 14:52:32Weiqi Li, Bin Chen, Jian ZhangAbstract
Mapping optimization algorithms into neural networks, deep unfolding networks (DUNs) have achieved impressive success in compressive sensing (CS). From the perspective of optimization, DUNs inherit a well-defined and interpretable structure from iterative steps. However, from the viewpoint of neural network design, most existing DUNs are inherently established based on traditional image-domain unfolding, which takes one-channel images as inputs and outputs between adjacent stages, resulting in insufficient information transmission capability and inevitable loss of the image details. In this paper, to break the above bottleneck, we first propose a generalized dual-domain optimization framework, which is general for inverse imaging and integrates the merits of both (1) image-domain and (2) convolutional-coding-domain priors to constrain the feasible region in the solution space. By unfolding the proposed framework into deep neural networks, we further design a novel Dual-Domain Deep Convolutional Coding Network (D3C2-Net) for CS imaging with the capability of transmitting high-throughput feature-level image representation through all the unfolded stages. Experiments on natural and MR images demonstrate that our D3C2-Net achieves higher performance and better accuracy-complexity trade-offs than other state-of-the-arts.
Abstract (translated)
URL
https://arxiv.org/abs/2207.13560
PDF
