Contour
-
Unpaired Object-Level SAR-to-Optical Image Translation for Aircraft with Keypoints-Guided Diffusion Models
2025-03-25 16:05:49Ruixi You, Hecheng Jia, Feng XuAbstract
Synthetic Aperture Radar (SAR) imagery provides all-weather, all-day, and high-resolution imaging capabilities but its unique imaging mechanism makes interpretation heavily reliant on expert knowledge, limiting interpretability, especially in complex target tasks. Translating SAR images into optical images is a promising solution to enhance interpretation and support downstream tasks. Most existing research focuses on scene-level translation, with limited work on object-level translation due to the scarcity of paired data and the challenge of accurately preserving contour and texture details. To address these issues, this study proposes a keypoint-guided diffusion model (KeypointDiff) for SAR-to-optical image translation of unpaired aircraft targets. This framework introduces supervision on target class and azimuth angle via keypoints, along with a training strategy for unpaired data. Based on the classifier-free guidance diffusion architecture, a class-angle guidance module (CAGM) is designed to integrate class and angle information into the diffusion generation process. Furthermore, adversarial loss and consistency loss are employed to improve image fidelity and detail quality, tailored for aircraft targets. During sampling, aided by a pre-trained keypoint detector, the model eliminates the requirement for manually labeled class and azimuth information, enabling automated SAR-to-optical translation. Experimental results demonstrate that the proposed method outperforms existing approaches across multiple metrics, providing an efficient and effective solution for object-level SAR-to-optical translation and downstream tasks. Moreover, the method exhibits strong zero-shot generalization to untrained aircraft types with the assistance of the keypoint detector.
Abstract (translated)
合成孔径雷达（SAR）图像提供全天候、全时段和高分辨率的成像能力，但其独特的成像机制使得解释工作高度依赖专家知识，限制了可解释性，特别是在复杂的任务目标中。将SAR图像转换为光学图像是增强解释和支持下游任务的一种有前景的方法。现有的大多数研究集中在场景级别的翻译上，由于配对数据稀缺以及准确保留轮廓和纹理细节的挑战，物体级别翻译的研究相对较少。 为了应对这些问题，本研究提出了一种关键点引导扩散模型（KeypointDiff），用于未配对飞机目标的SAR到光学图像转换。该框架通过关键点在目标类别和方位角方面引入监督，并针对未配对数据设计了训练策略。基于无分类器指导的扩散架构，设计了一个类-角度引导模块（CAGM），将类别和角度信息整合到了扩散生成过程中。此外，对抗损失和一致性损失被用来提高图像的真实感和细节质量，特别适用于飞机目标。 在采样期间，在预训练的关键点检测器的帮助下，该模型消除了手动标注的类别和方位角信息需求，实现了自动化的SAR到光学转换。实验结果表明，所提出的方法在多个指标上优于现有方法，为物体级别的SAR到光学图像转换以及下游任务提供了高效且有效的解决方案。此外，在关键点检测器的支持下，该方法展示了对未训练飞机类型强大的零样本泛化能力。
URL
https://arxiv.org/abs/2503.19798
PDF
-
Video Anomaly Detection with Contours - A Study
2025-03-25 12:11:50Mia Siemon, Ivan Nikolov, Thomas B. Moeslund, Kamal NasrollahiAbstract
In Pose-based Video Anomaly Detection prior art is rooted on the assumption that abnormal events can be mostly regarded as a result of uncommon human behavior. Opposed to utilizing skeleton representations of humans, however, we investigate the potential of learning recurrent motion patterns of normal human behavior using 2D contours. Keeping all advantages of pose-based methods, such as increased object anonymization, the shift from human skeletons to contours is hypothesized to leave the opportunity to cover more object categories open for future research. We propose formulating the problem as a regression and a classification task, and additionally explore two distinct data representation techniques for contours. To further reduce the computational complexity of Pose-based Video Anomaly Detection solutions, all methods in this study are based on shallow Neural Networks from the field of Deep Learning, and evaluated on the three most prominent benchmark datasets within Video Anomaly Detection and their human-related counterparts, totaling six datasets. Our results indicate that this novel perspective on Pose-based Video Anomaly Detection marks a promising direction for future research.
Abstract (translated)
在基于姿势的视频异常检测中，现有技术通常假设异常事件主要源于不常见的行为。然而，与使用人体骨架表示不同的是，我们探索了利用2D轮廓学习正常人类行为的循环运动模式的可能性。尽管保留了基于姿势方法的所有优点（如增强的对象匿名化），从人体骨骼到轮廓的转变被认为为未来研究开放了涵盖更多对象类别的可能性。我们建议将该问题表述为回归和分类任务，并且还探索了两种不同的数据表示技术用于轮廓。 为了进一步减少基于姿势视频异常检测解决方案的计算复杂度，本研究中的所有方法均基于深度学习领域中的浅层神经网络，并在三个最突出的视频异常检测基准数据集及其与人类相关的对应数据集上进行了评估，总计六个数据集。我们的结果表明，这种对基于姿势的视频异常检测的新视角为未来的研究开辟了有希望的方向。
URL
https://arxiv.org/abs/2503.19588
PDF
-
Pitch Contour Exploration Across Audio Domains: A Vision-Based Transfer Learning Approach
2025-03-24 21:33:13Jakob Abe{\ss}er, Simon Schw\"ar, Meinard M\"ullerAbstract
This study examines pitch contours as a unifying semantic construct prevalent across various audio domains including music, speech, bioacoustics, and everyday sounds. Analyzing pitch contours offers insights into the universal role of pitch in the perceptual processing of audio signals and contributes to a deeper understanding of auditory mechanisms in both humans and animals. Conventional pitch-tracking methods, while optimized for music and speech, face challenges in handling much broader frequency ranges and more rapid pitch variations found in other audio domains. This study introduces a vision-based approach to pitch contour analysis that eliminates the need for explicit pitch-tracking. The approach uses a convolutional neural network, pre-trained for object detection in natural images and fine-tuned with a dataset of synthetically generated pitch contours, to extract key contour parameters from the time-frequency representation of short audio segments. A diverse set of eight downstream tasks from four audio domains were selected to provide a challenging evaluation scenario for cross-domain pitch contour analysis. The results show that the proposed method consistently surpasses traditional techniques based on pitch-tracking on a wide range of tasks. This suggests that the vision-based approach establishes a foundation for comparative studies of pitch contour characteristics across diverse audio domains.
Abstract (translated)
这项研究探讨了音高轮廓作为一种普遍的语义结构，它贯穿于音乐、语音、生物声学和日常声音等多种音频领域。分析音高轮廓能够提供关于音高在感知处理音频信号中的通用作用的见解，并有助于加深对人类和动物听觉机制的理解。传统音高跟踪方法虽然针对音乐和语音进行了优化，但在应对其他音频领域中存在的更宽泛频率范围以及更快的音高变化时面临挑战。 该研究提出了一种基于视觉的方法来分析音高轮廓，这种方法消除了显式音高追踪的需求。这种方案利用了预先训练好的用于自然图像物体检测的卷积神经网络，并通过合成生成的音高轮廓数据集进行微调，从短音频片段的时间-频率表示中提取关键轮廓参数。 为了对跨领域音高轮廓分析提供一个具有挑战性的评估场景，选择了来自四个不同音频领域的八项下游任务。研究结果表明，所提出的方法在一系列广泛的任务上持续优于基于传统音高跟踪的技术。这暗示着视觉方法为比较研究提供了基础，即探讨多种音频领域中音高轮廓特征的特性。
URL
https://arxiv.org/abs/2503.19161
PDF
-
A Reliable and Efficient Detection Pipeline for Rodent Ultrasonic Vocalizations
2025-03-24 17:50:49Sabah Shahnoor Anis, Devin M. Kellis, Kris Ford Kaigler, Marlene A. Wilson, Christian O'ReillyAbstract
Analyzing ultrasonic vocalizations (USVs) is crucial for understanding rodents' affective states and social behaviors, but the manual analysis is time-consuming and prone to errors. Automated USV detection systems have been developed to address these challenges. Yet, these systems often rely on machine learning and fail to generalize effectively to new datasets. To tackle these shortcomings, we introduce ContourUSV, an efficient automated system for detecting USVs from audio recordings. Our pipeline includes spectrogram generation, cleaning, pre-processing, contour detection, post-processing, and evaluation against manual annotations. To ensure robustness and reliability, we compared ContourUSV with three state-of-the-art systems using an existing open-access USV dataset (USVSEG) and a second dataset we are releasing publicly along with this paper. On average, across the two datasets, ContourUSV outperformed the other three systems with a 1.51x improvement in precision, 1.17x in recall, 1.80x in F1 score, and 1.49x in specificity while achieving an average speedup of 117.07x.
Abstract (translated)
分析超声波发声（USVs）对于理解啮齿动物的情感状态和社会行为至关重要，但手动分析耗时且容易出错。为了解决这些问题，开发了自动化USV检测系统。然而，这些系统通常依赖于机器学习，并且在面对新数据集时泛化效果不佳。为了克服这些缺点，我们介绍了ContourUSV，这是一个高效的自动化系统，用于从音频记录中检测USVs。我们的流程包括光谱图生成、清理、预处理、轮廓检测、后期处理和与手动注释的评估。为了确保稳健性和可靠性，我们将ContourUSV与三个最先进的系统进行了比较，使用了一个现有的开放访问USV数据集（USVSEG）以及我们在本文中同时发布的第二个公开数据集。在两个数据集上平均而言，ContourUSV在精度方面提高了1.51倍，在召回率方面提高了1.17倍，在F1分数方面提高了1.80倍，在特异性方面提高了1.49倍，并且实现了117.07倍的平均加速。
URL
https://arxiv.org/abs/2503.18928
PDF
-
A Language Vision Model Approach for Automated Tumor Contouring in Radiation Oncology
2025-03-19 06:41:37Yi Luo, Hamed Hooshangnejad, Xue Feng, Gaofeng Huang, Xiaojian Chen, Rui Zhang, Quan Chen, Wil Ngwa, Kai DingAbstract
Background: Lung cancer ranks as the leading cause of cancer-related mortality worldwide. The complexity of tumor delineation, crucial for radiation therapy, requires expertise often unavailable in resource-limited settings. Artificial Intelligence(AI), particularly with advancements in deep learning (DL) and natural language processing (NLP), offers potential solutions yet is challenged by high false positive rates. Purpose: The Oncology Contouring Copilot (OCC) system is developed to leverage oncologist expertise for precise tumor contouring using textual descriptions, aiming to increase the efficiency of oncological workflows by combining the strengths of AI with human oversight. Methods: Our OCC system initially identifies nodule candidates from CT scans. Employing Language Vision Models (LVMs) like GPT-4V, OCC then effectively reduces false positives with clinical descriptive texts, merging textual and visual data to automate tumor delineation, designed to elevate the quality of oncology care by incorporating knowledge from experienced domain experts. Results: Deployments of the OCC system resulted in a significant reduction in the false discovery rate by 35.0%, a 72.4% decrease in false positives per scan, and an F1-score of 0.652 across our dataset for unbiased evaluation. Conclusions: OCC represents a significant advance in oncology care, particularly through the use of the latest LVMs to improve contouring results by (1) streamlining oncology treatment workflows by optimizing tumor delineation, reducing manual processes; (2) offering a scalable and intuitive framework to reduce false positives in radiotherapy planning using LVMs; (3) introducing novel medical language vision prompt techniques to minimize LVMs hallucinations with ablation study, and (4) conducting a comparative analysis of LVMs, highlighting their potential in addressing medical language vision challenges.
Abstract (translated)
背景：肺癌是全球癌症死亡的主要原因。肿瘤轮廓的复杂性对于放射治疗至关重要，但在资源有限的情况下，这种专业知识往往不可用。人工智能（AI），特别是深度学习（DL）和自然语言处理（NLP）的进步，提供了潜在解决方案，但面临高假阳性率的问题。 目的：我们开发了Oncology Contouring Copilot (OCC)系统，利用肿瘤学家的专业知识通过文本描述进行精确的肿瘤轮廓勾画，旨在通过结合AI的优势与人类监督来提高肿瘤学工作流程的效率。 方法：我们的OCC系统首先从CT扫描中识别出结节候选者。然后，它使用像GPT-4V这样的语言视觉模型（LVM）有效地利用临床描述性文本减少假阳性，并将文本和图像数据融合以自动化肿瘤勾画过程，旨在通过融入经验丰富的领域专家的知识来提升肿瘤学护理质量。 结果：OCC系统的部署在我们的数据集中显著降低了35.0%的错误发现率，每个扫描减少了72.4%的假阳性，并且F1分数为0.652。这证明了系统评估中的无偏性。 结论：OCC代表了肿瘤学护理的重要进展，特别是通过使用最新的LVM来改善轮廓绘制结果： (1) 通过优化肿瘤勾画过程，减少手动操作，从而简化肿瘤治疗工作流程； (2) 提供一个可扩展且直观的框架，利用LVM减少放射治疗计划中的假阳性； (3) 引入新的医疗语言视觉提示技术以最小化LVM幻觉，并进行消融研究验证； (4) 对LVM进行比较分析，突显其在解决医学语言视觉挑战方面的潜力。
URL
https://arxiv.org/abs/2503.14933
PDF
-
Diffusion-based Facial Aesthetics Enhancement with 3D Structure Guidance
2025-03-18 16:42:02Lisha Li, Jingwen Hou, Weide Liu, Yuming Fang, Jiebin YanAbstract
Facial Aesthetics Enhancement (FAE) aims to improve facial attractiveness by adjusting the structure and appearance of a facial image while preserving its identity as much as possible. Most existing methods adopted deep feature-based or score-based guidance for generation models to conduct FAE. Although these methods achieved promising results, they potentially produced excessively beautified results with lower identity consistency or insufficiently improved facial attractiveness. To enhance facial aesthetics with less loss of identity, we propose the Nearest Neighbor Structure Guidance based on Diffusion (NNSG-Diffusion), a diffusion-based FAE method that beautifies a 2D facial image with 3D structure guidance. Specifically, we propose to extract FAE guidance from a nearest neighbor reference face. To allow for less change of facial structures in the FAE process, a 3D face model is recovered by referring to both the matched 2D reference face and the 2D input face, so that the depth and contour guidance can be extracted from the 3D face model. Then the depth and contour clues can provide effective guidance to Stable Diffusion with ControlNet for FAE. Extensive experiments demonstrate that our method is superior to previous relevant methods in enhancing facial aesthetics while preserving facial identity.
Abstract (translated)
面部美学增强（FAE）的目标是通过调整面部图像的结构和外观来提高面部吸引力，同时尽可能地保留其身份特征。现有的大多数方法采用基于深度特征或评分的引导方式，用于生成模型进行FAE处理。尽管这些方法取得了令人满意的结果，但它们可能会产生过度美化的效果，导致身份一致性降低，或者对脸部美感的改善不足。为了在较少损失面部身份的情况下增强面部美学，我们提出了基于扩散的最近邻结构指导（NNSG-Diffusion）方法，这是一种利用3D结构引导来美化2D面部图像的扩散基础FAE方法。 具体而言，我们提出从最近的邻居参考脸中提取FAE指南。为了在FAE过程中减少面部结构的变化，通过参考匹配的2D参考脸和输入的2D人脸恢复出一个3D脸部模型，这样可以从该3D脸部模型中提取深度和轮廓指导信息。然后，这些深度和轮廓线索可以为Stable Diffusion结合ControlNet提供有效的指导，以进行FAE处理。 大量的实验表明，我们的方法在增强面部美学的同时保留面部身份方面优于以前的相关方法。
URL
https://arxiv.org/abs/2503.14402
PDF
-
HSOD-BIT-V2: A New Challenging Benchmarkfor Hyperspectral Salient Object Detection
2025-03-18 05:09:42Yuhao Qiu, Shuyan Bai, Tingfa Xu, Peifu Liu, Haolin Qin, Jianan LiAbstract
Salient Object Detection (SOD) is crucial in computer vision, yet RGB-based methods face limitations in challenging scenes, such as small objects and similar color features. Hyperspectral images provide a promising solution for more accurate Hyperspectral Salient Object Detection (HSOD) by abundant spectral information, while HSOD methods are hindered by the lack of extensive and available datasets. In this context, we introduce HSOD-BIT-V2, the largest and most challenging HSOD benchmark dataset to date. Five distinct challenges focusing on small objects and foreground-background similarity are designed to emphasize spectral advantages and real-world complexity. To tackle these challenges, we propose Hyper-HRNet, a high-resolution HSOD network. Hyper-HRNet effectively extracts, integrates, and preserves effective spectral information while reducing dimensionality by capturing the self-similar spectral features. Additionally, it conveys fine details and precisely locates object contours by incorporating comprehensive global information and detailed object saliency representations. Experimental analysis demonstrates that Hyper-HRNet outperforms existing models, especially in challenging scenarios.
Abstract (translated)
目标显著性检测（SOD）在计算机视觉领域至关重要，然而基于RGB的方法在处理小物体和颜色特征相似的复杂场景时存在局限。高光谱图像通过提供丰富的光谱信息为更准确的目标显著性检测（HSOD）提供了有前景的解决方案，但HSOD方法的发展受到可用数据集稀缺的影响。在此背景下，我们引入了HSOD-BIT-V2，这是迄今为止规模最大、挑战性最强的HSOD基准数据集。该数据集中设计了五个不同的挑战，重点关注小物体和前景背景相似度问题，以突出光谱信息的优势及现实世界的复杂性。为应对这些挑战，我们提出了高分辨率HSOD网络Hyper-HRNet。Hyper-HRNet能够有效提取、整合并保留有效的光谱信息，并通过捕获自我相似的光谱特征来降低维度。此外，该网络还通过融合全面的全局信息和详细的物体显著性表示来传达精细细节，并精确定位物体轮廓。实验分析表明，在具有挑战性的场景下，Hyper-HRNet优于现有的模型表现。
URL
https://arxiv.org/abs/2503.13906
PDF
-
Learning Contour-Guided 3D Face Reconstruction with Occlusions
Abstract
Recently, deep learning-based 3D face reconstruction methods have demonstrated promising advancements in terms of quality and efficiency. Nevertheless, these techniques face challenges in effectively handling occluded scenes and fail to capture intricate geometric facial details. Inspired by the principles of GANs and bump mapping, we have successfully addressed these issues. Our approach aims to deliver comprehensive 3D facial reconstructions, even in the presence of this http URL maintaining the overall shape's robustness, we introduce a mid-level shape refinement to the fundamental structure. Furthermore, we illustrate how our method adeptly extends to generate plausible details for obscured facial regions. We offer numerous examples that showcase the effectiveness of our framework in producing realistic results, where traditional methods often struggle. To substantiate the superior adaptability of our approach, we have conducted extensive experiments in the context of general 3D face reconstruction tasks, serving as concrete evidence of its regulatory prowess compared to manual occlusion removal methods.
Abstract (translated)
最近，基于深度学习的三维人脸重建方法在质量和效率方面取得了显著进展。然而，这些技术在处理被遮挡场景时仍然面临挑战，并且无法捕捉到复杂的几何面部细节。受GAN（生成对抗网络）和凹凸贴图原理的启发，我们成功地解决了这些问题。我们的方法旨在提供全面的三维面部重建，在存在遮挡的情况下也能保持整体形状的稳健性，并在此基础上引入了中层形状细化技术来改进基础结构。此外，我们展示了如何利用该方法为被遮盖的面部区域生成合理的细节。通过大量实例，我们证明了框架在产生逼真结果方面的有效性，而传统方法通常难以达到这一效果。 为了证实我们的方法具有更优越的适应性，我们在一般的三维人脸重建任务中进行了广泛的实验，并将其实验结果作为与手动遮挡去除方法相比，其调控能力更为卓越的具体证据。
URL
https://arxiv.org/abs/2503.12494
PDF
-
Spline refinement with differentiable rendering
2025-03-15 09:42:41Frans Zdyb, Albert Alonso, Julius B. KirkegaardAbstract
Detecting slender, overlapping structures remains a challenge in computational microscopy. While recent coordinate-based approaches improve detection, they often produce less accurate splines than pixel-based methods. We introduce a training-free differentiable rendering approach to spline refinement, achieving both high reliability and sub-pixel accuracy. Our method improves spline quality, enhances robustness to distribution shifts, and shrinks the gap between synthetic and real-world data. Being fully unsupervised, the method is a drop-in replacement for the popular active contour model for spline refinement. Evaluated on C. elegans nematodes, a popular model organism for drug discovery and biomedical research, we demonstrate that our approach combines the strengths of both coordinate- and pixel-based methods.
Abstract (translated)
在计算显微镜学中，检测纤细且重叠的结构仍然是一个挑战。尽管最近基于坐标的检测方法有所改进，但它们通常生成的质量不如基于像素的方法产生的样条曲线（spline）。我们引入了一种无需训练的可微渲染方法来优化样条曲线，从而实现高可靠性和亚像素精度。我们的方法提升了样条质量、增强了对分布变化的鲁棒性，并缩小了合成数据与真实世界数据之间的差距。该方法完全无监督，可以作为流行的主动轮廓模型中用于样条曲线细化部分的直接替代方案。在秀丽隐杆线虫（C. elegans）中的评估表明，我们的方法结合了基于坐标和基于像素的方法的优点，而秀丽隐杆线虫是药物发现和生物医学研究中的常用模式生物。
URL
https://arxiv.org/abs/2503.14525
PDF
-
Rethinking Rotation-Invariant Recognition of Fine-grained Shapes from the Perspective of Contour Points
2025-03-14 01:34:20Yanjie Xu, Handing Xu, Tianmu Wang, Yaguan Li, Yunzhi Chen, Zhenguo NieAbstract
Rotation-invariant recognition of shapes is a common challenge in computer vision. Recent approaches have significantly improved the accuracy of rotation-invariant recognition by encoding the rotational invariance of shapes as hand-crafted image features and introducing deep neural networks. However, the methods based on pixels have too much redundant information, and the critical geometric information is prone to early leakage, resulting in weak rotation-invariant recognition of fine-grained shapes. In this paper, we reconsider the shape recognition problem from the perspective of contour points rather than pixels. We propose an anti-noise rotation-invariant convolution module based on contour geometric aware for fine-grained shape recognition. The module divides the shape contour into multiple local geometric regions(LGA), where we implement finer-grained rotation-invariant coding in terms of point topological relations. We provide a deep network composed of five such cascaded modules for classification and retrieval experiments. The results show that our method exhibits excellent performance in rotation-invariant recognition of fine-grained shapes. In addition, we demonstrate that our method is robust to contour noise and the rotation centers. The source code is available at this https URL.
Abstract (translated)
旋转不变形状识别是计算机视觉中的一个常见挑战。近期的方法通过编码手工设计的图像特征来改进旋转不变性，这些特征描述了形状的旋转不变性，并引入了深度神经网络以提高准确性。然而，基于像素的方法包含了大量的冗余信息，关键几何信息容易过早丢失，这导致对细粒度形状进行弱旋转不变性识别的问题。 在本文中，我们重新考虑从轮廓点而非像素的角度来解决形状识别问题。为此，我们提出了一种抗噪的、基于轮廓几何感知的旋转不变卷积模块（Rotation-invariant Convolution Module with Contour Geometric Awareness, RC-MCGA），以用于细粒度形状识别。该模块将形状轮廓划分为多个局部几何区域(LGA)，并在点拓扑关系的基础上实现了更细化的旋转不变性编码。我们提供了一个由五个这样的级联模块组成的深度网络，用于分类和检索实验。实验结果表明，我们的方法在细粒度形状的旋转不变性识别中表现出色。 此外，我们还证明了该方法对轮廓噪声和旋转中心具有鲁棒性，并且源代码可在以下链接获得：[提供具体URL]。
URL
https://arxiv.org/abs/2503.10992
PDF
-
NAMI: Efficient Image Generation via Progressive Rectified Flow Transformers
2025-03-12 10:38:58Yuhang Ma, Bo Cheng, Shanyuan Liu, Ao Ma, Xiaoyu Wu, Liebucha Wu, Dawei Leng, Yuhui YinAbstract
Flow-based transformer models for image generation have achieved state-of-the-art performance with larger model parameters, but their inference deployment cost remains high. To enhance inference performance while maintaining generation quality, we propose progressive rectified flow transformers. We divide the rectified flow into different stages according to resolution, using fewer transformer layers at the low-resolution stages to generate image layouts and concept contours, and progressively adding more layers as the resolution increases. Experiments demonstrate that our approach achieves fast convergence and reduces inference time while ensuring generation quality. The main contributions of this paper are summarized as follows: (1) We introduce progressive rectified flow transformers that enable multi-resolution training, accelerating model convergence; (2) NAMI leverages piecewise flow and spatial cascading of Diffusion Transformer (DiT) to rapidly generate images, reducing inference time by 40% to generate a 1024 resolution image; (3) We propose NAMI-1K benchmark to evaluate human preference performance, aiming to mitigate distributional bias and prevent data leakage from open-source benchmarks. The results show that our model is competitive with state-of-the-art models.
Abstract (translated)
基于流的变压器模型在图像生成方面已经取得了最先进的性能，但随着模型参数的增大，其推理部署成本仍然很高。为了在保持生成质量的同时提高推理性能，我们提出了渐进式修正流变压器。我们将修正流根据分辨率划分为不同的阶段，在低分辨率阶段使用较少的变压器层来生成图像布局和概念轮廓，并逐步增加更多的层次以适应分辨率的增长。实验表明，我们的方法实现了快速收敛并减少了推理时间，同时确保了生成质量。 本文的主要贡献总结如下： 1. 我们介绍了渐进式修正流变压器，该模型支持多分辨率训练，加速了模型的收敛过程。 2. NAMI利用分段流和Diffusion Transformer (DiT)的空间级联来快速生成图像，在不降低生成质量的前提下将推理时间减少了40%，从而可以在1024分辨率下更快地生成图像。 3. 我们提出了NAMI-1K基准测试，以评估人类偏好的性能，并旨在减轻分布偏差并防止开源基准中的数据泄漏。结果表明，我们的模型在与现有最先进的模型竞争时具有竞争力。 通过这些方法和技术的结合使用，我们不仅提高了图像生成的速度和效率，还确保了模型能够更好地服务于各种应用场景的需求。
URL
https://arxiv.org/abs/2503.09242
PDF
-
Evaluation of state-of-the-art deep learning models in the segmentation of the heart ventricles in parasternal short-axis echocardiograms
2025-03-12 00:33:01Julian Rene Cuellar Buritica, Vu Dinh, Manjula Burri, Julie Roelandts, James Wendling, Jon D. KlingensmithAbstract
Previous studies on echocardiogram segmentation are focused on the left ventricle in parasternal long-axis views. In this study, deep-learning models were evaluated on the segmentation of the ventricles in parasternal short-axis echocardiograms (PSAX-echo). Segmentation of the ventricles in complementary echocardiogram views will allow the computation of important metrics with the potential to aid in diagnosing cardio-pulmonary diseases and other cardiomyopathies. Evaluating state-of-the-art models with small datasets can reveal if they improve performance on limited data. PSAX-echo were performed on 33 volunteer women. An experienced cardiologist identified end-diastole and end-systole frames from 387 scans, and expert observers manually traced the contours of the cardiac structures. Traced frames were pre-processed and used to create labels to train 2 specific-domain (Unet-Resnet101 and Unet-ResNet50), and 4 general-domain (3 Segment Anything (SAM) variants, and the Detectron2) deep-learning models. The performance of the models was evaluated using the Dice similarity coefficient (DSC), Hausdorff distance (HD), and difference in cross-sectional area (DCSA). The Unet-Resnet101 model provided superior performance in the segmentation of the ventricles with 0.83, 4.93 pixels, and 106 pixel2 on average for DSC, HD, and DCSA respectively. A fine-tuned MedSAM model provided a performance of 0.82, 6.66 pixels, and 1252 pixel2, while the Detectron2 model provided 0.78, 2.12 pixels, and 116 pixel2 for the same metrics respectively. Deep-learning models are suitable for the segmentation of the left and right ventricles in PSAX-echo. This study demonstrated that specific-domain trained models such as Unet-ResNet provide higher accuracy for echo segmentation than general-domain segmentation models when working with small and locally acquired datasets.
Abstract (translated)
先前关于心超分割的研究主要集中在胸骨旁长轴视图中的左心室。本研究评估了深度学习模型在胸骨旁短轴心超（PSAX-echo）中对心室分割的效果。通过不同视角的心超图像进行心脏结构的分割，有助于计算重要指标，并可能用于诊断心血管和肺部疾病及其他心肌病。使用小型数据集来评价最先进的模型性能可以揭示它们是否能够在有限的数据下提高表现。 本研究在33名志愿者女性中进行了PSAX-echo检查，一名有经验的心脏病专家从387个扫描图像中确定了舒张末期和收缩末期的帧，并由专业观察员手动描记心脏结构轮廓。预处理后的描记图用于训练两种特定领域（Unet-Resnet101 和 Unet-ResNet50）以及四种通用领域的深度学习模型（三种Segment Anything (SAM) 变体和Detectron2）。使用Dice相似系数(DSC)，Hausdorff距离(HD) 和面积差异(DCSA)来评估这些模型的性能。 Unet-Resnet101 模型在心室分割中表现出优越性，其平均DSC、HD和DCSA分别为0.83、4.93像素和106平方像素。微调后的MedSAM模型分别以0.82的DSC、6.66像素的HD以及1252平方像素的DCSA表现，而Detectron2模型则为0.78的DSC、2.12像素的HD和116平方像素的DCSA。 研究表明深度学习模型适合用于PSAX-echo中左右心室的分割。本研究证明，在处理小规模且本地采集的数据集时，特定领域训练的Unet-ResNet等模型比通用领域的分割模型具有更高的准确度。
URL
https://arxiv.org/abs/2503.08970
PDF
-
Structural and Statistical Texture Knowledge Distillation and Learning for Segmentation
2025-03-11 04:49:25Deyi Ji, Feng Zhao, Hongtao Lu, Feng Wu, Jieping YeAbstract
Low-level texture feature/knowledge is also of vital importance for characterizing the local structural pattern and global statistical properties, such as boundary, smoothness, regularity, and color contrast, which may not be well addressed by high-level deep features. In this paper, we aim to re-emphasize the low-level texture information in deep networks for semantic segmentation and related knowledge distillation tasks. To this end, we take full advantage of both structural and statistical texture knowledge and propose a novel Structural and Statistical Texture Knowledge Distillation (SSTKD) framework for semantic segmentation. Specifically, Contourlet Decomposition Module (CDM) is introduced to decompose the low-level features with iterative Laplacian pyramid and directional filter bank to mine the structural texture knowledge, and Texture Intensity Equalization Module (TIEM) is designed to extract and enhance the statistical texture knowledge with the corresponding Quantization Congruence Loss (QDL). Moreover, we propose the Co-occurrence TIEM (C-TIEM) and generic segmentation frameworks, namely STLNet++ and U-SSNet, to enable existing segmentation networks to harvest the structural and statistical texture information more effectively. Extensive experimental results on three segmentation tasks demonstrate the effectiveness of the proposed methods and their state-of-the-art performance on seven popular benchmark datasets, respectively.
Abstract (translated)
低级纹理特征/知识对于描述局部结构模式和全局统计特性（如边界、平滑度、规则性和颜色对比）至关重要，而这些特性可能无法通过高级深度特征很好地处理。在本文中，我们旨在重新强调在语义分割及相关知识蒸馏任务中的低级纹理信息的重要性。为此，我们充分利用了结构化和统计化的纹理知识，并提出了一种新颖的结构性与统计性纹理知识蒸馏（SSTKD）框架用于语义分割。 具体来说，引入了轮廓分解模块(CDM)，通过迭代拉普拉斯金字塔和方向滤波器组来分解低级特征，以挖掘结构化纹理知识；设计了纹理强度均衡模块(TIEM)并通过相应的量化一致性损失(QDL)提取并增强统计性纹理知识。此外，我们还提出了共现TIEM (C-TIEM) 和通用分割框架（即STLNet++和U-SSNet），使现有的分割网络能够更有效地获取结构化与统计性的纹理信息。 在三个分割任务上进行的大量实验结果验证了所提出方法的有效性及其在七个流行基准数据集上的先进性能。
URL
https://arxiv.org/abs/2503.08043
PDF
-
Text-IRSTD: Leveraging Semantic Text to Promote Infrared Small Target Detection in Complex Scenes
2025-03-10 12:33:07Feng Huang, Shuyuan Zheng, Zhaobing Qiu, Huanxian Liu, Huanxin Bai, Liqiong ChenAbstract
Infrared small target detection is currently a hot and challenging task in computer vision. Existing methods usually focus on mining visual features of targets, which struggles to cope with complex and diverse detection scenarios. The main reason is that infrared small targets have limited image information on their own, thus relying only on visual features fails to discriminate targets and interferences, leading to lower detection performance. To address this issue, we introduce a novel approach leveraging semantic text to guide infrared small target detection, called Text-IRSTD. It innovatively expands classical IRSTD to text-guided IRSTD, providing a new research idea. On the one hand, we devise a novel fuzzy semantic text prompt to accommodate ambiguous target categories. On the other hand, we propose a progressive cross-modal semantic interaction decoder (PCSID) to facilitate information fusion between texts and images. In addition, we construct a new benchmark consisting of 2,755 infrared images of different scenarios with fuzzy semantic textual annotations, called FZDT. Extensive experimental results demonstrate that our method achieves better detection performance and target contour recovery than the state-of-the-art methods. Moreover, proposed Text-IRSTD shows strong generalization and wide application prospects in unseen detection scenarios. The dataset and code will be publicly released after acceptance of this paper.
Abstract (translated)
红外小目标检测目前是计算机视觉领域的一个热点和具有挑战性的任务。现有方法通常侧重于挖掘目标的视觉特征，这在面对复杂多样的检测场景时显得力不从心。主要原因在于红外小目标自身图像信息有限，仅仅依赖视觉特征难以区分目标与干扰物，导致检测性能较低。为解决这一问题，我们提出了一种利用语义文本引导红外小目标检测的新方法，称为Text-IRSTD（文本辅助的红外小目标检测）。该方法创新性地将传统的IRSTD扩展到文本指导下的IRSTD，提供了一种新的研究思路。 一方面，我们设计了一种新颖的模糊语义文本提示来适应模棱两可的目标类别。另一方面，我们提出了一个逐步跨模态语义交互解码器（PCSID），以促进文本和图像之间的信息融合。此外，我们构建了一个包含2,755张不同场景下红外图像的新基准数据集，并使用模糊语义文本注释标记为FZDT。 广泛的实验结果表明，我们的方法在检测性能和目标轮廓恢复方面优于现有最先进的方法。另外，所提出的Text-IRSTD表现出较强的一般化能力和广泛的应用前景，在未见过的检测场景中亦有卓越表现。该数据集和代码将在论文接受后公开发布。
URL
https://arxiv.org/abs/2503.07249
PDF
-
CAFusion: Controllable Anatomical Synthesis of Perirectal Lymph Nodes via SDF-guided Diffusion
2025-03-10 04:59:54Weidong Guo, Hantao Zhang, Shouhong Wan, Bingbing Zou, Wanqin Wang, Chenyang Qiu, Peiquan JinAbstract
Lesion synthesis methods have made significant progress in generating large-scale synthetic datasets. However, existing approaches predominantly focus on texture synthesis and often fail to accurately model masks for anatomically complex lesions. Additionally, these methods typically lack precise control over the synthesis process. For example, perirectal lymph nodes, which range in diameter from 1 mm to 10 mm, exhibit irregular and intricate contours that are challenging for current techniques to replicate faithfully. To address these limitations, we introduce CAFusion, a novel approach for synthesizing perirectal lymph nodes. By leveraging Signed Distance Functions (SDF), CAFusion generates highly realistic 3D anatomical structures. Furthermore, it offers flexible control over both anatomical and textural features by decoupling the generation of morphological attributes (such as shape, size, and position) from textural characteristics, including signal intensity. Experimental results demonstrate that our synthetic data substantially improve segmentation performance, achieving a 6.45% increase in the Dice coefficient. In the visual Turing test, experienced radiologists found it challenging to distinguish between synthetic and real lesions, highlighting the high degree of realism and anatomical accuracy achieved by our approach. These findings validate the effectiveness of our method in generating high-quality synthetic lesions for advancing medical image processing applications.
Abstract (translated)
病灶合成方法在生成大规模合成数据集方面取得了显著进展。然而，现有方法主要集中在纹理合成上，并且通常无法准确建模解剖结构复杂的病灶掩码。此外，这些方法往往缺乏对合成过程的精确控制能力。例如，直肠旁淋巴结直径范围从1毫米到10毫米不等，其轮廓不规则且复杂，现有的技术很难忠实复制这种特征。为了解决这些问题，我们引入了一种名为CAFusion的新方法来合成直肠旁淋巴结。通过利用符号距离函数（SDF），CAFusion可以生成高度逼真的三维解剖结构。此外，它还提供了对解剖和纹理特性进行灵活控制的能力，将形态属性的生成（如形状、大小和位置）与包括信号强度在内的纹理特征相分离。实验结果显示，我们合成的数据显著提高了分割性能，在Dice系数上实现了6.45%的增长。在视觉图灵测试中，经验丰富的放射科医生难以区分合成病灶和真实病灶，这突显了我们的方法所达到的现实性和解剖准确性水平。这些发现验证了我们在生成高质量合成病变方面的方法的有效性，以推进医学图像处理应用的发展。
URL
https://arxiv.org/abs/2503.06919
PDF
-
Separability Membrane: 3D Active Contour for Point Cloud Surface Reconstruction
2025-03-07 08:15:02Gulpi Qorik Oktagalu Pratamasunu, Guoqing Hao, Kazuhiro FukuiAbstract
This paper proposes Separability Membrane, a robust 3D active contour for extracting a surface from 3D point cloud object. Our approach defines the surface of a 3D object as the boundary that maximizes the separability of point features, such as intensity, color, or local density, between its inner and outer regions based on Fisher's ratio. Separability Membrane identifies the exact surface of a 3D object by maximizing class separability while controlling the rigidity of the 3D surface model with an adaptive B-spline surface that adjusts its properties based on the local and global separability. A key advantage of our method is its ability to accurately reconstruct surface boundaries even when they are ambiguous due to noise or outliers, without requiring any training data or conversion to volumetric representation. Evaluations on a synthetic 3D point cloud dataset and the 3DNet dataset demonstrate the membrane's effectiveness and robustness under diverse conditions.
Abstract (translated)
本文提出了一种名为“Separability Membrane”的鲁棒三维活动轮廓方法，用于从三维点云对象中提取表面。我们的方法将三维物体的表面定义为一个边界，该边界最大化了内侧和外侧区域之间基于费希尔比率（Fisher's ratio）计算出的点特征（如强度、颜色或局部密度）的可分离性。Separability Membrane 通过最大限度地提高类别间的可分性和利用自适应B样条曲面控制三维表面模型的刚度来精确识别三维物体的实际表面，该B样条曲面可以根据局部和全局可分离性的不同调整其属性。 我们的方法的一个关键优势在于它能够在噪声或异常值导致边界模糊的情况下准确重建表面边界，并且不需要任何训练数据或转换为体积表示。在合成三维点云数据集以及3DNet数据集上的评估证明了该膜的有效性和鲁棒性，即使是在各种不同的条件下也是如此。
URL
https://arxiv.org/abs/2503.05217
PDF
-
Rice Grain Size Measurement using Image Processing
2025-03-05 06:16:13Ankush Tyagi, Dhruv Motwani, Vipul K. Dabhi, Harshadkumar B. PrajapatiAbstract
The rice grain quality can be determined from its size and chalkiness. The traditional approach to measure the rice grain size involves manual inspection, which is inefficient and leads to inconsistent results. To address this issue, an image processing based approach is proposed and developed in this research. The approach takes image of rice grains as input and outputs the number of rice grains and size of each rice grain. The different steps, such as extraction of region of interest, segmentation of rice grains, and sub-contours removal, involved in the proposed approach are discussed. The approach was tested on rice grain images captured from different height using mobile phone camera. The obtained results show that the proposed approach successfully detected 95\% of the rice grains and achieved 90\% accuracy for length and width measurement.
Abstract (translated)
稻谷的质量可以通过其大小和垩白度来确定。传统上，测量稻谷尺寸的方法是手动检查，这种方法效率低下且结果不一致。为了解决这一问题，在本次研究中提出并开发了一种基于图像处理的方法。该方法以稻谷的图片作为输入，并输出稻谷的数量以及每粒稻谷的大小。文中讨论了所提出的方案涉及的不同步骤，包括感兴趣区域的提取、稻谷的分割和次轮廓的移除等。 该方法在使用手机摄像头从不同高度拍摄的稻谷图像上进行了测试。获得的结果表明，所提出的方法成功检测出了95%的稻谷，并且在长度和宽度测量方面达到了90%的准确率。
URL
https://arxiv.org/abs/2503.03214
PDF
-
An Approach for Air Drawing Using Background Subtraction and Contour Extraction
Abstract
In this paper, we propose a novel approach for air drawing that uses image processing techniques to draw on the screen by moving fingers in the air. This approach benefits a wide range of applications such as sign language, in-air drawing, and 'writing' in the air as a new way of input. The approach starts with preparing ROI (Region of Interest) background images by taking a running average in initial camera frames and later subtracting it from the live camera frames to get a binary mask image. We calculate the pointer's position as the top of the contour on the binary image. When drawing a circle on the canvas in that position, it simulates the drawing. Furthermore, we combine the pre-trained Tesseract model for OCR purposes. To address the false contours, we perform hand detection based on the haar cascade before performing the background subtraction. In an experimental setup, we achieved a latency of only 100ms in air drawing. The code used to this research are available in GitHub as this https URL
Abstract (translated)
在这篇论文中，我们提出了一种新颖的空中绘图方法，该方法利用图像处理技术，在空中移动手指即可在屏幕上作画。这种方法适用于广泛的应用场景，如手语、空中绘画和作为输入新方式的“空中书写”。我们的方法首先通过初始摄像帧中的运行平均值准备ROI（感兴趣区域）背景图像，并随后从实时摄像帧中减去该背景以获得二进制掩模图像。我们计算指针的位置为二值图上轮廓的顶部。当在该位置画布上的相应位置绘制圆形时，它模拟了绘画的过程。此外，我们将预训练的Tesseract模型结合用于OCR（光学字符识别）目的。为了应对误判轮廓的问题，在执行背景减法之前进行了基于Haar级联的手部检测。在一个实验设置中，我们的空中绘图技术实现了仅100毫秒的延迟。 本研究使用的代码可在GitHub上的以下链接获取：[请在此处插入实际URL]
URL
https://arxiv.org/abs/2503.01497
PDF
-
Convex Hull-based Algebraic Constraint for Visual Quadric SLAM
2025-03-03 07:30:07Xiaolong Yu, Junqiao Zhao, Shuangfu Song, Zhongyang Zhu, Zihan Yuan, Chen Ye, Tiantian FengAbstract
Using Quadrics as the object representation has the benefits of both generality and closed-form projection derivation between image and world spaces. Although numerous constraints have been proposed for dual quadric reconstruction, we found that many of them are imprecise and provide minimal improvements to this http URL scrutinizing the existing constraints, we introduce a concise yet more precise convex hull-based algebraic constraint for object landmarks, which is applied to object reconstruction, frontend pose estimation, and backend bundle this http URL constraint is designed to fully leverage precise semantic segmentation, effectively mitigating mismatches between complex-shaped object contours and dual this http URL on public datasets demonstrate that our approach is applicable to both monocular and RGB-D SLAM and achieves improved object mapping and localization than existing quadric SLAM methods. The implementation of our method is available at this https URL.
Abstract (translated)
使用四元体（Quadrics）作为物体表示方法，兼具通用性和从图像空间到世界空间的闭式投影推导的优势。尽管已经提出了许多用于双曲面重建的约束条件，但我们发现其中很多并不精确，并且对现有方法改进甚微。通过仔细审查现有的约束条件，我们引入了一种简洁但更准确的基于凸包代数约束的方法来处理物体特征点的问题，这种方法适用于物体重建、前端姿态估计和后端捆绑调整。该约束条件旨在充分利用精确的语义分割信息，有效缓解复杂形状物体轮廓与双曲面之间的不匹配问题。在公共数据集上的实验表明，我们的方法既适用于单目视觉定位（SLAM）也适用于RGB-D SLAM，并且在物体映射和定位方面比现有的四元体SLAM方法表现更佳。我们方法的实现代码可以在以下网址获取：[此链接应为实际提供的GitHub或其他公开代码库地址，在这里以this https URL表示]。
URL
https://arxiv.org/abs/2503.01254
PDF
-
Sketch-1-to-3: One Single Sketch to 3D Detailed Face Reconstruction
2025-02-25 04:58:17Liting Wen, Zimo Yang, Xianlin Zhang, Chi Ding, Yue Zhang, Mingdao Wang, Xueming LiAbstract
3D face reconstruction from a single sketch is a critical yet underexplored task with significant practical applications. The primary challenges stem from the substantial modality gap between 2D sketches and 3D facial structures, including: (1) accurately extracting facial keypoints from 2D sketches; (2) preserving diverse facial expressions and fine-grained texture details; and (3) training a high-performing model with limited data. In this paper, we propose Sketch-1-to-3, a novel framework for realistic 3D face reconstruction from a single sketch, to address these challenges. Specifically, we first introduce the Geometric Contour and Texture Detail (GCTD) module, which enhances the extraction of geometric contours and texture details from facial sketches. Additionally, we design a deep learning architecture with a domain adaptation module and a tailored loss function to align sketches with the 3D facial space, enabling high-fidelity expression and texture reconstruction. To facilitate evaluation and further research, we construct SketchFaces, a real hand-drawn facial sketch dataset, and Syn-SketchFaces, a synthetic facial sketch dataset. Extensive experiments demonstrate that Sketch-1-to-3 achieves state-of-the-art performance in sketch-based 3D face reconstruction.
Abstract (translated)
从单幅素描进行三维人脸重建是一项具有重要实际应用价值但研究较少的任务。该任务的主要挑战在于二维素描与三维面部结构之间的显著模态差异，具体包括：（1）准确提取二维素描中的面部关键点；（2）保留多样化的面部表情和精细的纹理细节；以及（3）在数据有限的情况下训练高性能模型。为此，在本文中我们提出了Sketch-1-to-3框架，这是一种全新的从单幅素描生成逼真的三维人脸重建的方法，旨在解决上述挑战。 具体而言，我们首先引入了几何轮廓与纹理细节(GCTD)模块，该模块能够增强从面部素描中提取几何轮廓和纹理细节的能力。此外，为了使素描与其对应的3D面部空间对齐，我们设计了一个深度学习架构，其中包含领域适应模块及定制的损失函数，从而实现高保真度的表情与纹理重建。 为促进评估以及进一步的研究工作，我们构建了SketchFaces数据集，该数据集中包含了真实的素描人脸，并且还建立了一个合成的人脸素描数据集Syn-SketchFaces。通过广泛的实验表明，Sketch-1-to-3在基于素描的三维人脸识别任务上达到了最先进的性能水平。
URL
https://arxiv.org/abs/2502.17852
PDF
