Contour
-
Contour Flow Constraint: Preserving Global Shape Similarity for Deep Learning based Image Segmentation
2025-04-13 00:34:47Shengzhe Chen, Zhaoxuan Dong, Jun LiuAbstract
For effective image segmentation, it is crucial to employ constraints informed by prior knowledge about the characteristics of the areas to be segmented to yield favorable segmentation outcomes. However, the existing methods have primarily focused on priors of specific properties or shapes, lacking consideration of the general global shape similarity from a Contour Flow (CF) perspective. Furthermore, naturally integrating this contour flow prior image segmentation model into the activation functions of deep convolutional networks through mathematical methods is currently unexplored. In this paper, we establish a concept of global shape similarity based on the premise that two shapes exhibit comparable contours. Furthermore, we mathematically derive a contour flow constraint that ensures the preservation of global shape similarity. We propose two implementations to integrate the constraint with deep neural networks. Firstly, the constraint is converted to a shape loss, which can be seamlessly incorporated into the training phase for any learning-based segmentation framework. Secondly, we add the constraint into a variational segmentation model and derive its iterative schemes for solution. The scheme is then unrolled to get the architecture of the proposed CFSSnet. Validation experiments on diverse datasets are conducted on classic benchmark deep network segmentation models. The results indicate a great improvement in segmentation accuracy and shape similarity for the proposed shape loss, showcasing the general adaptability of the proposed loss term regardless of specific network architectures. CFSSnet shows robustness in segmenting noise-contaminated images, and inherent capability to preserve global shape similarity.
Abstract (translated)
为了实现有效的图像分割，利用先前知识中关于待分割区域特征的约束以获得良好的分割结果至关重要。然而，现有的方法主要集中在特定属性或形状的先验信息上，而缺乏从轮廓流（Contour Flow, CF）角度考虑的一般全局形状相似性的考量。此外，通过数学方法将这种基于轮廓流的先验图像分割模型自然地整合到深度卷积网络的激活函数中尚未被探索。 在本文中，我们建立了一种基于两个形状具有可比性轮廓这一前提的全球形状相似度的概念，并从数学上推导出一种保持全局形状相似性的轮廓流约束。我们提出了两种将该约束与深层神经网络结合的方法：首先，我们将约束转换为一个形状损失函数，它可以无缝地融入任何学习型分割框架的训练阶段；其次，我们在变分分割模型中添加了约束，并为其推导出了求解方案。随后，我们展开了这个方案以获得提出的CFSSnet架构的设计。 在经典基准深度网络分割模型上对不同数据集进行了验证实验。结果显示，所提出的形状损失极大地提高了分割的准确性和形状相似性，展示了无论具体网络结构如何，该损失项具有普遍适应性的特点。此外，CFSSnet在处理噪声污染图像时表现出强大的鲁棒性，并且具备保持全局形状相似性的内在能力。
URL
https://arxiv.org/abs/2504.09384
PDF
-
Enhanced Cooperative Perception Through Asynchronous Vehicle to Infrastructure Framework with Delay Mitigation for Connected and Automated Vehicles
2025-04-10 23:48:22Nithish Kumar Saravanan, Varun Jammula, Yezhou Yang, Jeffrey Wishart, Junfeng ZhaoAbstract
Perception is a key component of Automated vehicles (AVs). However, sensors mounted to the AVs often encounter blind spots due to obstructions from other vehicles, infrastructure, or objects in the surrounding area. While recent advancements in planning and control algorithms help AVs react to sudden object appearances from blind spots at low speeds and less complex scenarios, challenges remain at high speeds and complex intersections. Vehicle to Infrastructure (V2I) technology promises to enhance scene representation for AVs in complex intersections, providing sufficient time and distance to react to adversary vehicles violating traffic rules. Most existing methods for infrastructure-based vehicle detection and tracking rely on LIDAR, RADAR or sensor fusion methods, such as LIDAR-Camera and RADAR-Camera. Although LIDAR and RADAR provide accurate spatial information, the sparsity of point cloud data limits its ability to capture detailed object contours of objects far away, resulting in inaccurate 3D object detection results. Furthermore, the absence of LIDAR or RADAR at every intersection increases the cost of implementing V2I technology. To address these challenges, this paper proposes a V2I framework that utilizes monocular traffic cameras at road intersections to detect 3D objects. The results from the roadside unit (RSU) are then combined with the on-board system using an asynchronous late fusion method to enhance scene representation. Additionally, the proposed framework provides a time delay compensation module to compensate for the processing and transmission delay from the RSU. Lastly, the V2I framework is tested by simulating and validating a scenario similar to the one described in an industry report by Waymo. The results show that the proposed method improves the scene representation and the AV's perception range, giving enough time and space to react to adversary vehicles.
Abstract (translated)
感知是自动驾驶车辆（AV）的关键组成部分。然而，安装在AV上的传感器经常会因为其他车辆、基础设施或周围物体的遮挡而产生盲点。尽管最近规划和控制算法的进步有助于AV在低速和简单场景下应对从盲点突然出现的对象，但在高速度和复杂交叉口的情况下仍面临挑战。车到基础设施（V2I）技术有望提升复杂交叉口中自动驾驶车辆的场景表示能力，为遵守交通规则的对手车辆提供足够的反应时间和距离。目前大多数基于基础设施的车辆检测和跟踪方法依赖于激光雷达、雷达或传感器融合技术，如激光雷达-摄像头和雷达-摄像头组合。虽然激光雷达和雷达能提供精确的空间信息，但由于点云数据稀疏性，远距离物体细节轮廓捕捉能力有限，导致三维目标检测结果不准确。此外，每个交叉口都配备激光雷达或雷达会增加实施V2I技术的成本。 为解决这些挑战，本文提出了一种基于单目交通摄像头在道路交叉口进行3D对象检测的V2I框架。该框架通过异步延迟融合方法将路边单元（RSU）的结果与车载系统结合，以增强场景表示能力。此外，还提出了一个时间延迟补偿模块来弥补RSU处理和传输延迟。最后，在模拟并验证类似于Waymo行业报告中描述的情况下后测试了V2I框架。结果表明，该方法能改善场景表示能力和AV的感知范围，为对手车辆提供足够的反应时间和空间。
URL
https://arxiv.org/abs/2504.08172
PDF
-
Synthetic CT Generation from Time-of-Flight Non-Attenutaion-Corrected PET for Whole-Body PET Attenuation Correction
2025-04-10 04:49:41Weijie Chen, James Wang, Alan McMillanAbstract
Positron Emission Tomography (PET) imaging requires accurate attenuation correction (AC) to account for photon loss due to tissue density variations. In PET/MR systems, computed tomography (CT), which offers a straightforward estimation of AC is not available. This study presents a deep learning approach to generate synthetic CT (sCT) images directly from Time-of-Flight (TOF) non-attenuation corrected (NAC) PET images, enhancing AC for PET/MR. We first evaluated models pre-trained on large-scale natural image datasets for a CT-to-CT reconstruction task, finding that the pre-trained model outperformed those trained solely on medical datasets. The pre-trained model was then fine-tuned using an institutional dataset of 35 TOF NAC PET and CT volume pairs, achieving the lowest mean absolute error (MAE) of 74.49 HU and highest peak signal-to-noise ratio (PSNR) of 28.66 dB within the body contour region. Visual assessments demonstrated improved reconstruction of both bone and soft tissue structures from TOF NAC PET images. This work highlights the effectiveness of using pre-trained deep learning models for medical image translation tasks. Future work will assess the impact of sCT on PET attenuation correction and explore additional neural network architectures and datasets to further enhance performance and practical applications in PET imaging.
Abstract (translated)
正电子发射断层扫描（PET）成像需要准确的衰减校正（AC），以补偿由于组织密度变化引起的光子损失。在PET/MR系统中，提供直接衰减校正估计的计算机断层扫描（CT）不可用。这项研究提出了一种深度学习方法，可以从时间飞行（TOF）未校正衰减（NAC）PET图像生成合成CT（sCT）图像，从而增强PET/MR中的衰减校正。 首先，我们评估了在大规模自然图像数据集上预训练的模型，在CT到CT重建任务中进行测试。结果显示，这些预训练模型的表现优于仅在医学数据集上训练的模型。随后，通过使用一个包含35对TOF NAC PET和CT体体积配对的数据集对该预训练模型进行了微调。结果表明，该模型实现了最低均方绝对误差（MAE）74.49 HU以及最高的峰值信噪比（PSNR）28.66 dB的性能，在身体轮廓区域。 视觉评估显示，从TOF NAC PET图像中重建出的骨骼和软组织结构得到了显著改善。这项工作证明了使用预训练深度学习模型在医学图像转换任务中的有效性。未来的工作将评估sCT对PET衰减校正的影响，并探索其他神经网络架构和数据集以进一步提高性能和实际应用价值，特别是在PET成像领域。
URL
https://arxiv.org/abs/2504.07450
PDF
-
A Streamable Neural Audio Codec with Residual Scalar-Vector Quantization for Real-Time Communication
2025-04-09 03:49:00Xiao-Hang Jiang, Yang Ai, Rui-Chen Zheng, Zhen-Hua LingAbstract
This paper proposes StreamCodec, a streamable neural audio codec designed for real-time communication. StreamCodec adopts a fully causal, symmetric encoder-decoder structure and operates in the modified discrete cosine transform (MDCT) domain, aiming for low-latency inference and real-time efficient generation. To improve codebook utilization efficiency and compensate for the audio quality loss caused by structural causality, StreamCodec introduces a novel residual scalar-vector quantizer (RSVQ). The RSVQ sequentially connects scalar quantizers and improved vector quantizers in a residual manner, constructing coarse audio contours and refining acoustic details, respectively. Experimental results confirm that the proposed StreamCodec achieves decoded audio quality comparable to advanced non-streamable neural audio codecs. Specifically, on the 16 kHz LibriTTS dataset, StreamCodec attains a ViSQOL score of 4.30 at 1.5 kbps. It has a fixed latency of only 20 ms and achieves a generation speed nearly 20 times real-time on a CPU, with a lightweight model size of just 7M parameters, making it highly suitable for real-time communication applications.
Abstract (translated)
本文提出了一种名为StreamCodec的流式神经音频编解码器，专门用于实时通信。StreamCodec采用完全因果关系的对称编码-解码结构，并在修改后的离散余弦变换（MDCT）域中运行，旨在实现低延迟推理和实时高效生成。为了提高代码簿利用率并补偿由结构性因果性引起的音质损失，StreamCodec引入了一种新型残差标量向量量化器（RSVQ）。该RSVQ以残差方式顺序连接标量量化器和改进的向量量化器，分别构建粗略音频轮廓和细化声学细节。实验结果证实，所提出的StreamCodec达到了与先进的非流式神经音频编解码器相当的解码音质。具体而言，在16kHz LibriTTS数据集上，当比特率为1.5kbps时，StreamCodec获得了4.30的ViSQOL得分。它具有固定的20毫秒延迟，并且在CPU上的生成速度接近于实时的20倍，模型大小仅为7M参数，非常适合用于实时通信应用。
URL
https://arxiv.org/abs/2504.06561
PDF
-
Real-Time Pitch/F0 Detection Using Spectrogram Images and Convolutional Neural Networks
Abstract
This paper presents a novel approach to detect F0 through Convolutional Neural Networks and image processing techniques to directly estimate pitch from spectrogram images. Our new approach demonstrates a very good detection accuracy; a total of 92% of predicted pitch contours have strong or moderate correlations to the true pitch contours. Furthermore, the experimental comparison between our new approach and other state-of-the-art CNN methods reveals that our approach can enhance the detection rate by approximately 5% across various Signal-to-Noise Ratio conditions.
Abstract (translated)
本文提出了一种通过卷积神经网络和图像处理技术直接从频谱图中估算基频的新方法。我们的新方法展示了非常高的检测精度；预测的92%的音高轮廓与真实的音高轮廓具有强或中等的相关性。此外，我们新的方法与其他最先进的CNN方法之间的实验对比表明，在各种信噪比条件下，我们的方法可以提高约5%的检测率。
URL
https://arxiv.org/abs/2504.06165
PDF
-
Fast Sphericity and Roundness approximation in 2D and 3D using Local Thickness
2025-04-08 08:40:50Pawel Tomasz Pieta, Peter Winkel Rasumssen, Anders Bjorholm Dahl, Anders Nymark ChristensenAbstract
Sphericity and roundness are fundamental measures used for assessing object uniformity in 2D and 3D images. However, using their strict definition makes computation costly. As both 2D and 3D microscopy imaging datasets grow larger, there is an increased demand for efficient algorithms that can quantify multiple objects in large volumes. We propose a novel approach for extracting sphericity and roundness based on the output of a local thickness algorithm. For sphericity, we simplify the surface area computation by modeling objects as spheroids/ellipses of varying lengths and widths of mean local thickness. For roundness, we avoid a complex corner curvature determination process by approximating it with local thickness values on the contour/surface of the object. The resulting methods provide an accurate representation of the exact measures while being significantly faster than their existing implementations.
Abstract (translated)
球形度和圆度是评估二维和三维图像中对象均匀性的重要指标。然而，根据其严格的定义进行计算会非常耗时。随着二维和三维显微成像数据集的不断增大，对能够高效量化大量体积内多个物体特性的算法需求也在增加。我们提出了一种基于局部厚度算法输出来提取球形度和圆度的新方法。 对于球形度，我们简化了表面积计算过程，将对象建模为不同长度和宽度（以平均局部厚度为准）的椭球体/椭圆形，从而提高了效率。在处理圆度时，为了避免复杂的角落曲率确定过程，我们将它近似为物体轮廓或表面上的局部厚度值。这两种方法能够提供与现有算法相同准确性的结果，但计算速度显著提高。 这种方法不仅适用于二维图像中的对象，也能高效应用于三维成像数据中的复杂结构分析，在处理大量体积和多个物体的场景下尤为有效。
URL
https://arxiv.org/abs/2504.05808
PDF
-
kNN-SVC: Robust Zero-Shot Singing Voice Conversion with Additive Synthesis and Concatenation Smoothness Optimization
2025-04-08 04:59:56Keren Shao, Ke Chen, Matthew Baas, Shlomo DubnovAbstract
Robustness is critical in zero-shot singing voice conversion (SVC). This paper introduces two novel methods to strengthen the robustness of the kNN-VC framework for SVC. First, kNN-VC's core representation, WavLM, lacks harmonic emphasis, resulting in dull sounds and ringing artifacts. To address this, we leverage the bijection between WavLM, pitch contours, and spectrograms to perform additive synthesis, integrating the resulting waveform into the model to mitigate these issues. Second, kNN-VC overlooks concatenative smoothness, a key perceptual factor in SVC. To enhance smoothness, we propose a new distance metric that filters out unsuitable kNN candidates and optimize the summing weights of the candidates during inference. Although our techniques are built on the kNN-VC framework for implementation convenience, they are broadly applicable to general concatenative neural synthesis models. Experimental results validate the effectiveness of these modifications in achieving robust SVC. Demo: this http URL Code: this https URL
Abstract (translated)
在零样本歌声转换（Singing Voice Conversion，简称SVC）中，鲁棒性至关重要。本文介绍了两种新颖的方法来增强kNN-VC框架的鲁棒性。首先，kNN-VC的核心表示WavLM缺乏谐波强调，导致声音单调且有振铃效应。为了解决这个问题，我们利用了WavLM、音高轮廓和频谱图之间的双射关系来进行加法合成，并将生成的波形整合进模型中以减轻这些问题。 其次，kNN-VC忽略了连贯平滑性，这是SVC中的关键感知因素之一。为了增强这种平滑度，我们提出了一种新的距离度量方法来过滤掉不合适的k近邻候选者，并在推理过程中优化这些候选者的加权和。尽管我们的技术基于kNN-VC框架以方便实现，但它们广泛适用于一般的连贯神经合成模型。 实验结果验证了这些修改在实现鲁棒性SVC方面的有效性。 演示：[此处填写链接] 代码：[此处填写链接]
URL
https://arxiv.org/abs/2504.05686
PDF
-
Contour Integration Underlies Human-Like Vision
2025-04-07 16:45:06Ben Lonnqvist, Elsa Scialom, Abdulkadir Gokce, Zehra Merchant, Michael H. Herzog, Martin SchrimpfAbstract
Despite the tremendous success of deep learning in computer vision, models still fall behind humans in generalizing to new input distributions. Existing benchmarks do not investigate the specific failure points of models by analyzing performance under many controlled conditions. Our study systematically dissects where and why models struggle with contour integration -- a hallmark of human vision -- by designing an experiment that tests object recognition under various levels of object fragmentation. Humans (n=50) perform at high accuracy, even with few object contours present. This is in contrast to models which exhibit substantially lower sensitivity to increasing object contours, with most of the over 1,000 models we tested barely performing above chance. Only at very large scales ($\sim5B$ training dataset size) do models begin to approach human performance. Importantly, humans exhibit an integration bias -- a preference towards recognizing objects made up of directional fragments over directionless fragments. We find that not only do models that share this property perform better at our task, but that this bias also increases with model training dataset size, and training models to exhibit contour integration leads to high shape bias. Taken together, our results suggest that contour integration is a hallmark of object vision that underlies object recognition performance, and may be a mechanism learned from data at scale.
Abstract (translated)
尽管深度学习在计算机视觉领域取得了巨大成功，但模型仍然无法像人类一样将知识泛化到新的输入分布中。现有的基准测试没有通过分析多种受控条件下的性能来具体调查模型的失败点。我们的研究系统地分析了模型为何以及在哪方面难以处理轮廓整合——这是人类视觉的一个标志性特征——我们设计了一个实验，该实验在不同程度的对象碎片化情况下测试物体识别能力。50名参与者（n=50）即使在仅存在少量对象轮廓的情况下也能以高准确率完成任务，这与大多数深度学习模型形成了鲜明对比，后者对增加的轮廓敏感度显著降低，我们在超过1,000个模型中进行测试时发现，大多数表现甚至低于随机猜测水平。只有当训练数据集规模达到非常大的程度（约5B）时，这些模型才开始接近人类的表现。 更重要的是，人类表现出一种整合偏见——倾向于识别由定向碎片构成的对象而非无方向性的碎片。我们发现不仅那些具有这种性质的模型在我们的任务中表现得更好，而且随着训练数据集规模的增加，这种偏向性也在增强，并且通过训练使模型展示轮廓整合会导致高形状偏好。 综上所述，我们的研究结果表明，轮廓整合是物体视觉的一个标志特征，对对象识别性能起着关键作用，并可能是从大规模数据中学习到的一种机制。
URL
https://arxiv.org/abs/2504.05253
PDF
-
MSA-UNet3+: Multi-Scale Attention UNet3+ with New Supervised Prototypical Contrastive Loss for Coronary DSA Image Segmentation
2025-04-07 15:35:30Rayan Merghani Ahmed, Adnan Iltaf, Bin Li, Shoujun ZhouAbstract
The accurate segmentation of coronary Digital Subtraction Angiography (DSA) images is essential for diagnosing and treating coronary artery diseases. Despite advances in deep learning-based segmentation, challenges such as low contrast, noise, overlapping structures, high intra-class variance, and class imbalance limit precise vessel delineation. To overcome these limitations, we propose the MSA-UNet3+: a Multi-Scale Attention enhanced UNet3+ architecture for coronary DSA image segmentation. The framework combined Multi-Scale Dilated Bottleneck (MSD-Bottleneck) with Contextual Attention Fusion Module (CAFM), which not only enhances multi-scale feature extraction but also preserve fine-grained details, and improve contextual understanding. Furthermore, we propose a new Supervised Prototypical Contrastive Loss (SPCL), which combines supervised and prototypical contrastive learning to minimize class imbalance and high intra-class variance by focusing on hard-to-classified background samples. Experiments carried out on a private coronary DSA dataset demonstrate that MSA-UNet3+ outperforms state-of-the-art methods, achieving a Dice coefficient of 87.73%, an F1-score of 87.78%, and significantly reduced Average Surface Distance (ASD) and Average Contour Distance (ACD). The developed framework provides clinicians with precise vessel segmentation, enabling accurate identification of coronary stenosis and supporting informed diagnostic and therapeutic decisions. The code will be released at the following GitHub profile link this https URL.
Abstract (translated)
冠状数字减影血管造影（DSA）图像的精确分割对于诊断和治疗冠状动脉疾病至关重要。尽管基于深度学习的分割技术取得了进展，但诸如对比度低、噪声大、结构重叠、高类内变化以及类别不平衡等问题仍然限制了对血管的准确描绘。为克服这些局限性，我们提出了一种改进的多尺度注意力增强UNet3+架构（MSA-UNet3+），专门用于冠状DSA图像分割。该框架结合了多尺度膨胀瓶颈模块（MSD-Bottleneck）与上下文注意力融合模块（CAFM），不仅增强了多尺度特征提取，还保留了精细细节，并提升了上下文理解能力。此外，我们提出了一种新的监督原型对比损失函数（SPCL），它将监督学习和原型对比学习相结合，通过关注难以分类的背景样本来最小化类别不平衡及高类内变化。 在私人冠状DSA数据集上的实验表明，MSA-UNet3+优于现有方法，在Dice系数上达到了87.73%，F1分数为87.78%，同时显著减少了平均表面距离（ASD）和平均轮廓距离（ACD）。开发的这一框架为临床医生提供了精确的血管分割结果，有助于准确识别冠状动脉狭窄，并支持诊断与治疗决策。相关代码将在以下GitHub链接发布：[此URL]。
URL
https://arxiv.org/abs/2504.05184
PDF
-
Edge Approximation Text Detector
2025-04-05 00:12:51Chuang Yang, Xu Han, Tao Han, Han Han, Bingxuan Zhao, Qi WangAbstract
Pursuing efficient text shape representations helps scene text detection models focus on compact foreground regions and optimize the contour reconstruction steps to simplify the whole detection pipeline. Current approaches either represent irregular shapes via box-to-polygon strategy or decomposing a contour into pieces for fitting gradually, the deficiency of coarse contours or complex pipelines always exists in these models. Considering the above issues, we introduce EdgeText to fit text contours compactly while alleviating excessive contour rebuilding processes. Concretely, it is observed that the two long edges of texts can be regarded as smooth curves. It allows us to build contours via continuous and smooth edges that cover text regions tightly instead of fitting piecewise, which helps avoid the two limitations in current models. Inspired by this observation, EdgeText formulates the text representation as the edge approximation problem via parameterized curve fitting functions. In the inference stage, our model starts with locating text centers, and then creating curve functions for approximating text edges relying on the points. Meanwhile, truncation points are determined based on the location features. In the end, extracting curve segments from curve functions by using the pixel coordinate information brought by truncation points to reconstruct text contours. Furthermore, considering the deep dependency of EdgeText on text edges, a bilateral enhanced perception (BEP) module is designed. It encourages our model to pay attention to the recognition of edge features. Additionally, to accelerate the learning of the curve function parameters, we introduce a proportional integral loss (PI-loss) to force the proposed model to focus on the curve distribution and avoid being disturbed by text scales.
Abstract (translated)
追求高效的文本形状表示有助于场景文本检测模型聚焦于紧致的前景区域，并优化轮廓重建步骤以简化整个检测流程。当前的方法要么通过盒到多边形策略来表示不规则形状，要么将一个轮廓分解成多个片段逐步拟合。这些方法中往往存在粗略轮廓或复杂流程的问题。 为了解决上述问题，我们提出了EdgeText模型，旨在紧凑地匹配文本轮廓同时缓解过度的轮廓重建过程。具体而言，观察到文本的两条长边可以视为平滑曲线，这使得我们可以构建连续且平滑覆盖文本区域的边缘以代替分段拟合的方式，从而避免现有模型中的两个局限性。受到这一发现启发，EdgeText将文本表示形式化为参数化曲线拟合函数的问题。 在推理阶段，我们的模型首先定位文字中心，然后基于这些点创建用于近似文字边界的曲线函数。同时，根据位置特征确定截断点。最后，利用由截断点带来的像素坐标信息从曲线函数中提取曲线段来重建文本轮廓。 此外，考虑到EdgeText对文本边缘的深度依赖性，我们设计了一种双边增强感知（BEP）模块，以鼓励模型注意边缘特性的识别。为了加速学习曲线函数参数的速度，我们引入了比例积分损失（PI-loss），使提出的模型专注于曲线分布并避免受到文字尺度的影响。
URL
https://arxiv.org/abs/2504.04001
PDF
-
Dynamic Objective MPC for Motion Planning of Seamless Docking Maneuvers
2025-04-04 09:05:05Oliver Schumann, Michael Buchholz, Klaus DietmayerAbstract
Automated vehicles and logistics robots must often position themselves in narrow environments with high precision in front of a specific target, such as a package or their charging station. Often, these docking scenarios are solved in two steps: path following and rough positioning followed by a high-precision motion planning algorithm. This can generate suboptimal trajectories caused by bad positioning in the first phase and, therefore, prolong the time it takes to reach the goal. In this work, we propose a unified approach, which is based on a Model Predictive Control (MPC) that unifies the advantages of Model Predictive Contouring Control (MPCC) with a Cartesian MPC to reach a specific goal pose. The paper's main contributions are the adaption of the dynamic weight allocation method to reach path ends and goal poses inside driving corridors, and the development of the so-called dynamic objective MPC. The latter is an improvement of the dynamic weight allocation method, which can inherently switch state-dependent from an MPCC to a Cartesian MPC to solve the path-following problem and the high-precision positioning tasks independently of the location of the goal pose seamlessly by one algorithm. This leads to foresighted, feasible, and safe motion plans, which can decrease the mission time and result in smoother trajectories.
Abstract (translated)
自动车辆和物流机器人常常需要在狭窄环境中精确地定位到特定目标，如包裹或充电站前。通常，这些对接场景的解决方法是分两步进行：首先是路径跟踪和粗略定位，然后使用高精度运动规划算法进行精确定位。这种方法可能由于初始阶段的定位不佳而生成次优轨迹，从而延长达到目标所需的时间。 本文提出了一种统一的方法，该方法基于模型预测控制（MPC），结合了轮廓模型预测控制（MPCC）与笛卡尔MPC的优势，以实现特定的目标姿态。论文的主要贡献在于将动态权重分配法应用于到达路径末端和驾驶走廊内的目标位置，并开发所谓的动态目标MPC。后者是对动态权重分配方法的改进，能够在状态依赖的情况下从MPCC无缝切换到笛卡尔MPC，独立于目标位置解决路径跟踪问题与高精度定位任务。这种方法能够产生预见性、可行性和安全性更高的运动计划，从而减少执行时间并生成更平滑的轨迹。
URL
https://arxiv.org/abs/2504.03280
PDF
-
Segmentation variability and radiomics stability for predicting Triple-Negative Breast Cancer subtype using Magnetic Resonance Imaging
2025-04-02 12:48:01Isabella Cama, Alejandro Guzm\'an, Cristina Campi, Michele Piana, Karim Lekadir, Sara Garbarino, Oliver D\'iazAbstract
Most papers caution against using predictive models for disease stratification based on unselected radiomic features, as these features are affected by contouring variability. Instead, they advocate for the use of the Intraclass Correlation Coefficient (ICC) as a measure of stability for feature selection. However, the direct effect of segmentation variability on the predictive models is rarely studied. This study investigates the impact of segmentation variability on feature stability and predictive performance in radiomics-based prediction of Triple-Negative Breast Cancer (TNBC) subtype using Magnetic Resonance Imaging. A total of 244 images from the Duke dataset were used, with segmentation variability introduced through modifications of manual segmentations. For each mask, explainable radiomic features were selected using the Shapley Additive exPlanations method and used to train logistic regression models. Feature stability across segmentations was assessed via ICC, Pearson's correlation, and reliability scores quantifying the relationship between feature stability and segmentation variability. Results indicate that segmentation accuracy does not significantly impact predictive performance. While incorporating peritumoral information may reduce feature reproducibility, it does not diminish feature predictive capability. Moreover, feature selection in predictive models is not inherently tied to feature stability with respect to segmentation, suggesting that an overreliance on ICC or reliability scores for feature selection might exclude valuable predictive features.
Abstract (translated)
大多数论文警告不要基于未选择的放射组学特征来使用预测模型进行疾病分层，因为这些特征会受到轮廓变异的影响。相反，它们提倡将组内相关系数（Intraclass Correlation Coefficient, ICC）作为特征选择稳定性的衡量标准。然而，分割变异性对预测模型的直接影响却很少被研究。这项研究探讨了分割变异性对基于放射影像预测三阴性乳腺癌（TNBC）亚型时特性稳定性和预测性能的影响。总共使用了杜克数据集中244张图像，并通过修改手动分割引入了分割变异性。对于每种掩模，利用Shapley Additive exPlanations方法选择可解释的放射组学特征并用于训练逻辑回归模型。通过对ICC、Pearson相关性以及量化特性稳定性和分割变异性的关系可靠评分进行评估，来评估不同分割下的特征稳定性。结果显示，分割准确性对预测性能没有显著影响。虽然纳入肿瘤周围的组织信息可能会降低特征再现性，但这并不会削弱其预测能力。此外，在预测模型中选择的特征并不必然与针对分割的稳定性相关联，这表明过于依赖ICC或可靠评分进行特征选择可能排除了有价值的预测特性。
URL
https://arxiv.org/abs/2504.01692
PDF
-
PolygoNet: Leveraging Simplified Polygonal Representation for Effective Image Classification
2025-04-01 22:05:00Salim Khazem, Jeremy Fix, C\'edric PradalierAbstract
Deep learning models have achieved significant success in various image related tasks. However, they often encounter challenges related to computational complexity and overfitting. In this paper, we propose an efficient approach that leverages polygonal representations of images using dominant points or contour coordinates. By transforming input images into these compact forms, our method significantly reduces computational requirements, accelerates training, and conserves resources making it suitable for real time and resource constrained applications. These representations inherently capture essential image features while filtering noise, providing a natural regularization effect that mitigates overfitting. The resulting lightweight models achieve performance comparable to state of the art methods using full resolution images while enabling deployment on edge devices. Extensive experiments on benchmark datasets validate the effectiveness of our approach in reducing complexity, improving generalization, and facilitating edge computing applications. This work demonstrates the potential of polygonal representations in advancing efficient and scalable deep learning solutions for real world scenarios. The code for the experiments of the paper is provided in this https URL.
Abstract (translated)
深度学习模型在各种与图像相关的任务中取得了显著的成功。然而，它们经常遇到计算复杂性和过拟合的挑战。在这篇论文中，我们提出了一种有效的方法，利用图像的多边形表示（使用主导点或轮廓坐标）。通过将输入图像转换为这些紧凑形式，我们的方法可以显着减少计算需求、加速训练并节省资源，使其适用于实时和资源受限的应用程序。这些表示自然地捕获了关键的图像特征，并过滤掉了噪声，提供了一种天然的正则化效应以减轻过拟合问题。所得到的轻量级模型在性能上与使用全分辨率图像的最佳方法相当，同时能够在边缘设备上部署。在基准数据集上的广泛实验验证了我们方法在降低复杂性、改进泛化和促进边缘计算应用方面的有效性。这项工作展示了多边形表示在为现实世界场景推进高效且可扩展的深度学习解决方案方面所具有的潜力。论文实验的代码可在以下网址获取：[提供的URL]。
URL
https://arxiv.org/abs/2504.01214
PDF
-
Contextualized Autonomous Drone Navigation using LLMs Deployed in Edge-Cloud Computing
2025-04-01 10:04:40Hongqian Chen, Yun Tang, Antonios Tsourdos, Weisi GuoAbstract
Autonomous navigation is usually trained offline in diverse scenarios and fine-tuned online subject to real-world experiences. However, the real world is dynamic and changeable, and many environmental encounters/effects are not accounted for in real-time due to difficulties in describing them within offline training data or hard to describe even in online scenarios. However, we know that the human operator can describe these dynamic environmental encounters through natural language, adding semantic context. The research is to deploy Large Language Models (LLMs) to perform real-time contextual code adjustment to autonomous navigation. The challenge not evaluated in literature is what LLMs are appropriate and where should these computationally heavy algorithms sit in the computation-communication edge-cloud computing architectures. In this paper, we evaluate how different LLMs can adjust both the navigation map parameters dynamically (e.g., contour map shaping) and also derive navigation task instruction sets. We then evaluate which LLMs are most suitable and where they should sit in future edge-cloud of 6G telecommunication architectures.
Abstract (translated)
自主导航通常在多样化的场景中离线训练，然后根据实际经验在线微调。然而，现实世界是动态且多变的，许多环境交互或影响由于难以描述而无法实时纳入离线训练数据中，甚至在在线场景下也难以准确描述。但是我们知道，人类操作员可以通过自然语言来描述这些动态的环境互动，并添加语义上下文信息。研究旨在部署大型语言模型（LLMs）以进行实时上下文代码调整，应用于自主导航系统。目前文献未评估的问题是：哪些LLM模型适合这一任务以及这些计算密集型算法应位于边缘-云计算架构中的哪个位置。 本文将评估不同的LLM如何动态调整导航地图参数（如地形图的形状），并生成导航任务指令集。然后我们将评估哪种LLM最适合这项工作，并确定它们在未来6G电信边缘-云架构中应该占据的位置。
URL
https://arxiv.org/abs/2504.00607
PDF
-
BoundMatch: Boundary detection applied to semi-supervised segmentation for urban-driving scenes
2025-03-30 17:02:26Haruya Ishikawa, Yoshimitsu AokiAbstract
Semi-supervised semantic segmentation (SS-SS) aims to mitigate the heavy annotation burden of dense pixel labeling by leveraging abundant unlabeled images alongside a small labeled set. While current teacher-student consistency regularization methods achieve strong results, they often overlook a critical challenge: the precise delineation of object boundaries. In this paper, we propose BoundMatch, a novel multi-task SS-SS framework that explicitly integrates semantic boundary detection into the consistency regularization pipeline. Our core mechanism, Boundary Consistency Regularized Multi-Task Learning (BCRM), enforces prediction agreement between teacher and student models on both segmentation masks and detailed semantic boundaries. To further enhance performance and sharpen contours, BoundMatch incorporates two lightweight fusion modules: Boundary-Semantic Fusion (BSF) injects learned boundary cues into the segmentation decoder, while Spatial Gradient Fusion (SGF) refines boundary predictions using mask gradients, leading to higher-quality boundary pseudo-labels. This framework is built upon SAMTH, a strong teacher-student baseline featuring a Harmonious Batch Normalization (HBN) update strategy for improved stability. Extensive experiments on diverse datasets including Cityscapes, BDD100K, SYNTHIA, ADE20K, and Pascal VOC show that BoundMatch achieves competitive performance against state-of-the-art methods while significantly improving boundary-specific evaluation metrics. We also demonstrate its effectiveness in realistic large-scale unlabeled data scenarios and on lightweight architectures designed for mobile deployment.
Abstract (translated)
半监督语义分割（SS-SS）旨在通过利用大量未标记的图像和少量已标注的数据集来减轻密集像素标签注释的工作负担。尽管当前的教师-学生一致性正则化方法取得了强大的成果，但它们往往忽视了一个关键挑战：精确地划分物体边界。在本文中，我们提出了一种名为BoundMatch的新颖多任务半监督语义分割框架，该框架明确将语义边界的检测集成到了一致性正则化的管道之中。我们的核心机制是边界一致性正则化多任务学习（BCRM），它要求教师和学生模型在分割掩码及详细的语义边界上达成预测的一致性。为了进一步提升性能并锐化轮廓，BoundMatch结合了两个轻量级融合模块：边界-语义融合（BSF）将学习到的边界线索注入到解码器中，而空间梯度融合（SGF）则利用掩码梯度细化边界预测，从而生成更高质量的边界伪标签。该框架基于SAMTH构建，这是一个强大的教师-学生基线模型，采用了和谐批量归一化（HBN）更新策略以提高稳定性。 在包括Cityscapes、BDD100K、SYNTHIA、ADE20K和Pascal VOC在内的多样化数据集上进行的广泛实验表明，BoundMatch取得了与最先进方法相当的表现，并显著提升了特定边界评价指标。此外，我们还展示了该框架在现实大规模未标注数据场景以及面向移动部署的轻量级架构上的有效性。
URL
https://arxiv.org/abs/2503.23519
PDF
-
The realization of tones in spontaneous spoken Taiwan Mandarin: a corpus-based survey and theory-driven computational modeling
2025-03-29 17:39:55Yuxin Lu, Yu-Ying Chuang, R. Harald BaayenAbstract
A growing body of literature has demonstrated that semantics can co-determine fine phonetic detail. However, the complex interplay between phonetic realization and semantics remains understudied, particularly in pitch realization. The current study investigates the tonal realization of Mandarin disyllabic words with all 20 possible combinations of two tones, as found in a corpus of Taiwan Mandarin spontaneous speech. We made use of Generalized Additive Mixed Models (GAMs) to model f0 contours as a function of a series of predictors, including gender, tonal context, tone pattern, speech rate, word position, bigram probability, speaker and word. In the GAM analysis, word and sense emerged as crucial predictors of f0 contours, with effect sizes that exceed those of tone pattern. For each word token in our dataset, we then obtained a contextualized embedding by applying the GPT-2 large language model to the context of that token in the corpus. We show that the pitch contours of word tokens can be predicted to a considerable extent from these contextualized embeddings, which approximate token-specific meanings in contexts of use. The results of our corpus study show that meaning in context and phonetic realization are far more entangled than standard linguistic theory predicts.
Abstract (translated)
越来越多的研究表明，语义可以共同决定精细的语音细节。然而，语音实现与语义之间的复杂互动关系仍然研究不足，特别是在音高的实现方面。本项研究调查了台湾普通话自然口语语料库中所有20种可能的双音节词（两个声调组合）的音高实现情况。我们利用广义加性混合模型（GAMs），将基频曲线建模为一系列预测变量的功能，包括性别、声调背景、声调模式、说话速度、单词位置、二元语法概率、说话人和具体词汇。在GAM分析中，单词和词义成为影响基频曲线的关键预测因素，其效果大小超过了声调模式的效应。对于数据集中每个词实例，我们通过将大型语言模型（如GPT-2）应用于该实例在其语料库中的上下文来获取其上下文嵌入。结果显示，这些上下文化的嵌入可以相当程度上预测单词实例的音高曲线，它们近似于在使用过程中特定词汇的具体意义。我们的语料库研究结果表明，在具体语言环境中意义与语音实现之间相互纠缠的程度远远超过标准语言理论所预测的范围。
URL
https://arxiv.org/abs/2503.23163
PDF
-
Image Decomposition with G-norm Weighted by Total Symmetric Variation
Abstract
In this paper, we propose a novel variational model for decomposing images into their respective cartoon and texture parts. Our model characterizes certain non-local features of any Bounded Variation (BV) image by its Total Symmetric Variation (TSV). We demonstrate that TSV is effective in identifying regional boundaries. Based on this property, we introduce a weighted Meyer's $G$-norm to identify texture interiors without including contour edges. For BV images with bounded TSV, we show that the proposed model admits a solution. Additionally, we design a fast algorithm based on operator-splitting to tackle the associated non-convex optimization problem. The performance of our method is validated by a series of numerical experiments.
Abstract (translated)
在这篇论文中，我们提出了一种新颖的变分模型，用于将图像分解为其卡通部分和纹理部分。我们的模型通过总对称变化（TSV）来描述任何有界变差（BV）图像中的某些非局部特征。我们证明了TSV在识别区域边界方面是有效的。基于这一特性，我们引入了一种加权的Meyer $G$-范数，以在不包含轮廓边缘的情况下标识纹理内部。对于具有有界TSV的BV图像，我们展示了所提出的模型存在解。此外，我们设计了一种基于算子分裂的快速算法来解决相关的非凸优化问题。通过一系列数值实验验证了我们方法的有效性。
URL
https://arxiv.org/abs/2503.22560
PDF
-
Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models
2025-03-25 16:05:49Ruixi You, Hecheng Jia, Feng XuAbstract
Synthetic Aperture Radar (SAR) imagery provides all-weather, all-day, and high-resolution imaging capabilities but its unique imaging mechanism makes interpretation heavily reliant on expert knowledge, limiting interpretability, especially in complex target tasks. Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks. Most existing research focuses on scene-level translation, with limited work on object-level translation due to the scarcity of paired data and the challenge of accurately preserving contour and texture details. To address these issues, this study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets. This framework introduces supervision on target class and azimuth angle via keypoints, along with a training strategy for unpaired data. Based on the classifier-free guidance diffusion architecture, a class-angle guidance module (CAGM) is designed to integrate class and angle information into the diffusion generation process. Furthermore, adversarial loss and consistency loss are employed to improve image fidelity and detail quality, tailored for aircraft targets. During sampling, aided by a pre-trained keypoint detector, the model eliminates the requirement for manually labeled class and azimuth information, enabling automated SAR-to-optical translation. Experimental results demonstrate that the proposed method outperforms existing approaches across multiple metrics, providing an efficient and effective solution for object-level SAR-to-optical translation and downstream tasks. Moreover, the method exhibits strong zero-shot generalization to untrained aircraft types with the assistance of the keypoint detector.
Abstract (translated)
合成孔径雷达（SAR）图像提供全天候、全时段和高分辨率的成像能力，但其独特的成像机制使得解释工作高度依赖专家知识，限制了可解释性，特别是在复杂的任务目标中。将SAR图像转换为光学图像是增强解释和支持下游任务的一种有前景的方法。现有的大多数研究集中在场景级别的翻译上，由于配对数据稀缺以及准确保留轮廓和纹理细节的挑战，物体级别翻译的研究相对较少。 为了应对这些问题，本研究提出了一种关键点引导扩散模型（KeypointDiff），用于未配对飞机目标的SAR到光学图像转换。该框架通过关键点在目标类别和方位角方面引入监督，并针对未配对数据设计了训练策略。基于无分类器指导的扩散架构，设计了一个类-角度引导模块（CAGM），将类别和角度信息整合到了扩散生成过程中。此外，对抗损失和一致性损失被用来提高图像的真实感和细节质量，特别适用于飞机目标。 在采样期间，在预训练的关键点检测器的帮助下，该模型消除了手动标注的类别和方位角信息需求，实现了自动化的SAR到光学转换。实验结果表明，所提出的方法在多个指标上优于现有方法，为物体级别的SAR到光学图像转换以及下游任务提供了高效且有效的解决方案。此外，在关键点检测器的支持下，该方法展示了对未训练飞机类型强大的零样本泛化能力。
URL
https://arxiv.org/abs/2503.19798
PDF
-
Video Anomaly Detection with Contours - A Study
2025-03-25 12:11:50Mia Siemon, Ivan Nikolov, Thomas B. Moeslund, Kamal NasrollahiAbstract
In Pose-based Video Anomaly Detection prior art is rooted on the assumption that abnormal events can be mostly regarded as a result of uncommon human behavior. Opposed to utilizing skeleton representations of humans, however, we investigate the potential of learning recurrent motion patterns of normal human behavior using 2D contours. Keeping all advantages of pose-based methods, such as increased object anonymization, the shift from human skeletons to contours is hypothesized to leave the opportunity to cover more object categories open for future research. We propose formulating the problem as a regression and a classification task, and additionally explore two distinct data representation techniques for contours. To further reduce the computational complexity of Pose-based Video Anomaly Detection solutions, all methods in this study are based on shallow Neural Networks from the field of Deep Learning, and evaluated on the three most prominent benchmark datasets within Video Anomaly Detection and their human-related counterparts, totaling six datasets. Our results indicate that this novel perspective on Pose-based Video Anomaly Detection marks a promising direction for future research.
Abstract (translated)
在基于姿势的视频异常检测中，现有技术通常假设异常事件主要源于不常见的行为。然而，与使用人体骨架表示不同的是，我们探索了利用2D轮廓学习正常人类行为的循环运动模式的可能性。尽管保留了基于姿势方法的所有优点（如增强的对象匿名化），从人体骨骼到轮廓的转变被认为为未来研究开放了涵盖更多对象类别的可能性。我们建议将该问题表述为回归和分类任务，并且还探索了两种不同的数据表示技术用于轮廓。 为了进一步减少基于姿势视频异常检测解决方案的计算复杂度，本研究中的所有方法均基于深度学习领域中的浅层神经网络，并在三个最突出的视频异常检测基准数据集及其与人类相关的对应数据集上进行了评估，总计六个数据集。我们的结果表明，这种对基于姿势的视频异常检测的新视角为未来的研究开辟了有希望的方向。
URL
https://arxiv.org/abs/2503.19588
PDF
-
Pitch Contour Exploration Across Audio Domains: A Vision-Based Transfer Learning Approach
2025-03-24 21:33:13Jakob Abe{\ss}er, Simon Schw\"ar, Meinard M\"ullerAbstract
This study examines pitch contours as a unifying semantic construct prevalent across various audio domains including music, speech, bioacoustics, and everyday sounds. Analyzing pitch contours offers insights into the universal role of pitch in the perceptual processing of audio signals and contributes to a deeper understanding of auditory mechanisms in both humans and animals. Conventional pitch-tracking methods, while optimized for music and speech, face challenges in handling much broader frequency ranges and more rapid pitch variations found in other audio domains. This study introduces a vision-based approach to pitch contour analysis that eliminates the need for explicit pitch-tracking. The approach uses a convolutional neural network, pre-trained for object detection in natural images and fine-tuned with a dataset of synthetically generated pitch contours, to extract key contour parameters from the time-frequency representation of short audio segments. A diverse set of eight downstream tasks from four audio domains were selected to provide a challenging evaluation scenario for cross-domain pitch contour analysis. The results show that the proposed method consistently surpasses traditional techniques based on pitch-tracking on a wide range of tasks. This suggests that the vision-based approach establishes a foundation for comparative studies of pitch contour characteristics across diverse audio domains.
Abstract (translated)
这项研究探讨了音高轮廓作为一种普遍的语义结构，它贯穿于音乐、语音、生物声学和日常声音等多种音频领域。分析音高轮廓能够提供关于音高在感知处理音频信号中的通用作用的见解，并有助于加深对人类和动物听觉机制的理解。传统音高跟踪方法虽然针对音乐和语音进行了优化，但在应对其他音频领域中存在的更宽泛频率范围以及更快的音高变化时面临挑战。 该研究提出了一种基于视觉的方法来分析音高轮廓，这种方法消除了显式音高追踪的需求。这种方案利用了预先训练好的用于自然图像物体检测的卷积神经网络，并通过合成生成的音高轮廓数据集进行微调，从短音频片段的时间-频率表示中提取关键轮廓参数。 为了对跨领域音高轮廓分析提供一个具有挑战性的评估场景，选择了来自四个不同音频领域的八项下游任务。研究结果表明，所提出的方法在一系列广泛的任务上持续优于基于传统音高跟踪的技术。这暗示着视觉方法为比较研究提供了基础，即探讨多种音频领域中音高轮廓特征的特性。
URL
https://arxiv.org/abs/2503.19161
PDF
