Abstract
Although deep neural networks have provided impressive gains in performance, these improvements often come at the cost of increased computational complexity and expense. In many cases, such as 3D volume or video classification tasks, not all slices or frames are necessary due to inherent redundancies. To address this issue, we propose a novel learnable subsampling framework that can be integrated into any neural network architecture. Subsampling, being a nondifferentiable operation, poses significant challenges for direct adaptation into deep learning models. While some works, have proposed solutions using the Gumbel-max trick to overcome the problem of non-differentiability, they fall short in a crucial aspect: they are only task-adaptive and not inputadaptive. Once the sampling mechanism is learned, it remains static and does not adjust to different inputs, making it unsuitable for real-world applications. To this end, we propose an attention-guided sampling module that adapts to inputs even during inference. This dynamic adaptation results in performance gains and reduces complexity in deep neural network models. We demonstrate the effectiveness of our method on 3D medical imaging datasets from MedMNIST3D as well as two ultrasound video datasets for classification tasks, one of them being a challenging in-house dataset collected under real-world clinical conditions.
Abstract (translated)
尽管深度神经网络在性能上取得了令人印象深刻的提升，但这些改进通常是以计算复杂性和成本增加为代价的。在许多情况下，例如3D体积或视频分类任务中，并非所有的切片或帧都是必要的，因为存在内在冗余性。为此，我们提出了一种新型的学习采样框架，它可以集成到任何神经网络架构中。作为非可微操作的采样，在直接适应深度学习模型时会带来显著挑战。尽管有些工作提出了使用Gumbel-max技巧来克服非可微性的难题，但它们在关键方面仍有不足：仅适用于特定任务，并不具备输入自适应性。一旦采样机制被学会后便保持静态，不能根据不同的输入进行调整，这使其不适合应用于现实世界中。 为了解决这个问题，我们提出了一种基于注意力引导的采样模块，在推理过程中也能根据输入数据进行动态调整。这种动态适配不仅带来了性能提升，还减少了深度神经网络模型中的复杂性。我们在MedMNIST3D的数据集中验证了我们的方法在3D医学成像上的有效性，同时也在两个超声视频分类任务的数据集上进行了实验，其中一个为一个具有挑战性的内部数据集，在真实的临床条件下收集。
URL
https://arxiv.org/abs/2510.12376
