Denoising
-
DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation
2026-08-10 08:41:28Wei Zhou, Wanyi Ning, Yinshang Guo, Qianxiao Fang, Haitao Qian, Yingpeng LiAbstract
Audio-visual speech enhancement under real-world conditions remains challenging due to unreliable visual inputs and the lack of large-scale training data with realistic acoustic conditions. Existing approaches usually fuse visual features directly into the separation network, making them vulnerable to degraded visual signals. In this paper, we present DAVE, a decoupled audio-visual enhancement framework for real-world speech separation. Firstly, to address the data scarcity issue, we construct DAVE-Corpus, a large-scale training corpus with 219,411 mixtures generated from public meeting corpora through combinatorial acoustic augmentation. Then, we introduce a progressive multi-objective optimization strategy to jointly improve speech separation, intelligibility, speaker identity preservation, and perceptual quality. We further develop a certified selective enhancement chain that applies scene routing, GAN-based denoising, and loudness normalization only within the no-reference partition, guaranteeing non-degradation of reference-based metrics. Experimental results on the Real-World Audio-Visual Speech Enhancement Challenge demonstrate the robustness of DAVE under both real-world mixed scenarios and visual degradation conditions.
Abstract (translated)
URL
https://arxiv.org/abs/2608.09288
PDF
-
BAG: Budget-Aware Gating for Diffusion Caching
2026-08-10 07:54:28Tong Zhao, Mingkun Lei, Yucheng Han, Chi ZhangAbstract
Diffusion caching is a lightweight strategy that accelerates Diffusion Transformers (DiTs) by reusing intermediate features across denoising steps, but existing paradigms face a fundamental trade-off: online heuristics lack global budget awareness, whereas static schedules lack instance adaptivity and fail to flexibly adapt to varying runtime budget constraints. To bridge this gap, we present BAG (Budget-Aware Gating), a novel caching policy that unifies global budget pacing with dynamic, instance-adaptive feature reuse. Rather than relying on hand-crafted rules, BAG employs a lightweight gating network that dynamically decides whether to execute a full computation or reuse cached features at each step by jointly conditioning on the budget state and local trajectory feedback. We train this policy via offline-to-online schedule distillation, transferring the decision-making of offline-searched schedules into a compact online gate. Extensive experiments on FLUX.1-dev and Wan2.1 demonstrate that BAG consistently outperforms state-of-the-art caching methods across various speedup tiers while remaining robust across different resolutions, seeds, and guidance scales. Code will be released.
Abstract (translated)
URL
https://arxiv.org/abs/2608.09231
PDF
-
Bright-Channel Retinex Enhancement with a Conditional Overdispered-Noise Analysis
Abstract
I present a training-free low-light enhancement method that combines local bright-channel illumination estimation, Retinex division, and edge-preserving denoising. For a fixed illumination estimate, a conditional Negative -Binominal psueduo-count method characterises the heteroscedastic noise amplified by division. The unconstrained reflectance ratio is the pixelwise maximum-likelihood estimate, with a boundary solution for zero-valued observations; the implemented estimate additionally applies illumination filtering and range clipping. The NB model is a diagnostic noise analysis rather than a calibrated sensor model, and the final fixed-bandwidth bilateral filter is an empirical approximation rather than the exact Bayesian solution. On the LOL-v1 dataset, the methodobtains mean PSNR/SSIM of 17.74dB/0.739, the highest values among the evaluated with conventional methods. A 400X600 image is processed at approximately 43 FPS on an Apple M2 Pro CPU.
Abstract (translated)
URL
https://arxiv.org/abs/2608.09137
PDF
-
DeepFreqMark: End-To-End Learnable Frequency-Domain Watermarking with Spherical Attack Simulation for Latent Diffusion Models
2026-08-10 01:36:22Chen-Hsiu Huang, Mario K\"oppen, Ja-Ling WuAbstract
The proliferation of AI-generated images produced by Latent Diffusion Models (LDMs) has raised critical concerns regarding copyright infringement and misinformation. Although existing frequency-domain watermarking methods embed handcrafted geometric patterns into the initial latent noise prior to generation, they suffer from limited capacity and rigid pattern designs. We propose DeepFreqMark, an end-to-end learnable frequency-domain watermarking framework that replaces manual pattern engineering with a neural message encoder and decoder. To circumvent the computational bottleneck caused by Denoising Diffusion Implicit Model (DDIM) inversion during training, we introduce a Spherical Linear Interpolation (Slerp)-based attack simulation. This approach operates directly on the noise latent while strictly preserving the Gaussian variance. Extensive experiments demonstrate that DeepFreqMark achieves significantly lower Bit Error Rates (BER) than baseline methods under real-world attacks and scales to 256 bits message capacity. Our source code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08999
PDF
-
Toward CT-Equivalent Image Quality in Low-Dose Radiotherapy Planning: Conditional Diffusion-Based CBCT-to-CT Synthesis and the Impact of CBCT Input Representation
2026-08-09 21:15:08Alzahra Altalib, Chunhui Li, Christopher Hamill Taylor, Sankar Pillai, Alessandro PerelliAbstract
During standard radiotherapy planning, repeated CT acquisitions are often required for patient registration, verification, and adaptive planning, resulting in increased cumulative X-ray dose. To mitigate this, low-dose cone-beam CT (CBCT) is routinely acquired during treatment delivery. However, CBCT image quality remains insufficient for accurate dose calculation and adaptive radiotherapy planning due to increased scatter, noise, beam hardening, and reconstruction related artifacts. This study develops a supervised deep learning based CBCT to CT synthesis framework using a conditional denoising diffusion probabilistic model (DDPM), where the generation of a CT-based planning for accurate positioning and dose calculation is obtained using generative models with low dose CBCT imaging. Beyond demonstrating CBCT to CT synthesis, the primary objective is to investigate how the representation of CBCT input data, either standard clinical DICOM CBCT images or filtered back-projection (FDK) reconstructions from raw projection data, affects the performance of diffusion based CT synthesis. The overarching aim is to assess whether physics aware CBCT representations better support CT-equivalent image quality while maintaining reduced imaging dose in radiotherapy workflows.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08919
PDF
-
IDATA: Scalable Invertible Diffusion for Unrestricted Adversarial Transfer Attack
2026-08-09 14:26:42Yi Pan, Jun-Jie Huang, Tianrui Liu, Zihan Chen, Lin Liu, Zhao WentaoAbstract
Unrestricted adversarial transfer attacks are important for evaluating the black-box robustness of deep visual models. Diffusion-based attacks have shown promising transferability and visual imperceptibility by optimizing adversarial perturbations along denoising trajectories in latent space. However, existing methods are limited by two challenges: memory-intensive multistep backpropagation and frequency-agnostic perturbation over intermediate latents. To address these issues, we propose IDATA, a memory-efficient diffusion framework for unrestricted adversarial transfer attack. IDATA consists of two key components: an Invertible Diffusion Module (IDM) and a Low-Frequency Constraint Module (LFCM). Specifically, IDM reformulates adversarial optimization over diffusion trajectories as an invertible process, enabling constant-memory backpropagation through on-demand reconstruction of intermediate states instead of storing the full denoising chain. Moreover, LFCM leverages Discrete Wavelet Transform (DWT) to decompose latent variables into low- and high-frequency components, restricting perturbations to semantically stable low-frequency subspaces, thereby improving transferability while preserving visual imperceptibility. Extensive experiments on multiple benchmarks and diverse model architectures demonstrate that IDATA consistently outperforms state-of-the-art baselines in attack success rate, memory efficiency, and visual imperceptibility. These results suggest that IDATA is a promising tool for black-box robustness evaluation of deep visual models. Code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08734
PDF
-
High-Quality Exposure Correction with Diffusion-Based Image Generation Priors
2026-08-09 14:05:08Ziwen Li, Meng Cao, Jinpu Zhang, Chunyang Li, Long Bao, Heng Sun, Yuehuan WangAbstract
Although most existing exposure correction methods achieve high fidelity, they often place excessive focus on overall pixel-wise accuracy, making it challenging to effectively model extreme exposure regions, which results in suboptimal perceptual quality. Recently, diffusion models have received significant attention due to their remarkable performance in the realm of image generation. However, their successful application to exposure correction remains a challenging and open question. The key challenge lies in generating accurate image structures and maintaining high image fidelity during stochastic diffusion processes. In this paper, we propose DPEC (Diffusion Prior-based Exposure Correction), a novel framework for image exposure correction that utilizes diffusion-based image generation priors encapsulated in pre-trained large-scale diffusion models. Specifically, we first propose an efficient fine-tuning strategy to derive an exposure corrector from pre-trained models, enabling the generation of enhanced images in a single-step denoising process. Moreover, we seamlessly combine the strengths of diffusion models and regression models, and design a joint cross-attention module to integrate multi-scale diffusion prior features, thereby effectively preserving high-frequency details and minimizing random artifacts. The diffusion model focuses on dealing with low-frequency content rather than all the intricate texture details. The experimental results demonstrate that the proposed DPEC method consistently outperforms existing state-of-the-art methods on multiple exposure correction datasets, whether in terms of fidelity, perceptual quality, or visual effects.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08720
PDF
-
FiRe: Fixed-Noise Refinement for Visual Counterfactual Explanations
2026-08-09 12:15:37Yan Zeng, Changlu Guo, Oskar Kristoffersen, Anders Nymark Christensen, Morten Rieger Hannemose, Anders Bjorholm DahlAbstract
Visual counterfactual explanations aim to change classifier decisions through realistic and localized edits while preserving decision-irrelevant content. Existing DDPM-based methods typically perform classifier-guided editing along a long reverse denoising trajectory. The changing noise levels make semantic editability and spatial control difficult to balance, and the editable state is noisy, whereas the target classifier is trained on clean images. As a result, these methods require either costly recursive denoising or low-quality one-step estimates to obtain classifier-facing clean images. We propose FiRe, a Fixed-noise Refinement framework for visual counterfactual explanations. Rather than following a reverse denoising trajectory, FiRe maps the input to a fixed noise level and iteratively refines the noisy state at that level. To provide clean images for classifier guidance, FiRe first adapts Pixel Mean Flow to visual counterfactual explanation, enabling direct clean-image prediction from noisy states. To make fixed-noise refinement produce minimal and localized counterfactual edits, FiRe introduces three FiRe-specific controls: a dynamic dual-mask strategy, adaptive guidance, and early stopping, which determine where edits accumulate, which changes become visible, and when refinement stops. Experiments on five tasks across three datasets show that, compared with the strongest recent baseline, FiRe achieves about 3$\times$ faster online inference and 8$\times$ fewer FLOPs while obtaining comparable or state-of-the-art counterfactual quality.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08664
PDF
-
SDDBMs: Soft Denoising Diffusion Bridge Models
2026-08-09 09:19:46Shiyi Qi, Kun He, Mingmou LiuAbstract
Diffusion bridge models leverage Doob's \(h\)-transform to construct stochastic transports between arbitrary endpoint distributions, and have shown strong potential in image-to-image translation and restoration. However, most existing bridge models rely on hard endpoint conditioning, which forces the terminal state to match a prescribed target exactly. This hard constraint induces terminal-boundary singularities: the terminal law collapses to a Dirac measure, and the resulting drift coefficients become ill-conditioned near the endpoint. In this paper, we propose Soft Denoising Diffusion Bridge Models (SDDBMs), a generalized framework that regularizes diffusion bridges directly at the level of their terminal constraints. Instead of imposing an exact endpoint, SDDBMs prescribe a non-degenerate Gaussian terminal marginal under the transformed path measure, with a flexible terminal center and variance. Starting from this prescribed marginal, we develop a complete closed-form construction of the soft bridge, including the Gaussian terminal reweighting and soft \(h\)-function, the induced Gaussian forward marginals and \(\mathbf{x}_0\)-free dynamics. Theoretically, SDDBMs provide a unified probabilistic perspective that encompasses existing diffusion bridge models, including DDBMs, GOUB, and UniDB, as special cases under specific parameter choices. Extensive experiments on image restoration tasks demonstrate that SDDBMs achieve improved numerical stability and superior generation quality over existing bridge-based methods.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08594
PDF
-
SC-Diff: Semantically Calibrated Diffusion for Visible-to-Infrared Image Translation
2026-08-09 07:57:40Junyin Zhang, Siyu Huang, Jianxiong Ye, Haowei Gong, Ruicheng Zhang, Deyu Meng, Chenqiang GaoAbstract
Visible-to-infrared image translation provides a practical way to expand infrared training data using abundant visible images. Diffusion models are promising for this task because of their strong generative performance. However, existing diffusion-based methods typically use semantic priors only as external conditions, without explicitly regulating token interactions within the denoising network. Consequently, they struggle to preserve object locations, shapes, and semantic layouts required for reliable annotation reuse. We propose SC-Diff, a semantically calibrated latent diffusion framework that uses semantic priors for both conditional guidance and internal self-attention calibration. A pretrained SAM3 model with predefined text prompts first extracts category-specific semantic masks from visible images. These masks are merged into a semantic map and fused with the visible image as the input condition. The same map is converted into token-level semantic labels to calibrate self-attention in the denoising network. Based on these labels, we introduce Semantic-Guided Self-Attention Calibration (SGSC), which adaptively applies positive biases to query-key pairs of the same category. The query-wise calibration strength depends on the dispersion of attention across semantic categories and the attention assigned to the query's own category. The original attention scores further modulate the bias, giving greater calibration to same-category keys with stronger responses. This soft calibration reduces cross-category interference while retaining global contextual interactions, thereby improving semantic consistency in generated infrared images. Extensive experiments show that SC-Diff improves perceptual quality and produces more effective synthetic training data for downstream infrared object detection.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08555
PDF
-
Wiener Representation Filtering for VLM Hallucination Suppression
2026-08-08 14:51:17Ameen Ali, Tamim Zoabi, Lidor Brami, Lior WolfAbstract
Vision-language models (VLMs) excel at open-ended captioning and visual QA but often describe objects, attributes, or relations absent from the image, a phenomenon known as object hallucination. We propose a {training-free, post-hoc representation editing technique} that operates in the representation space of the language backbone. The method performs a lightweight, one-time offline calibration on a modest paired dataset to estimate the required covariance structures, using only forward passes and empirical second-order statistics with no gradient updates or fine-tuning, after which the correction is absorbed directly into the model's existing weights. By modeling hidden states as a superposition of truthful and hallucination-associated components, we derive a Wiener-type estimator whose optimal gains are given in closed form from the covariances of paired truthful and hallucinated representations. An eigendecomposition yields mode-wise attenuation that respects a stability criterion, i.e., the filter responds continuously to estimation noise. The correction is applied once to the feed-forward output projections of selected deeper layers, at inference time, the model runs unchanged and at the same speed. Experiments on LLaVA-1.5, MiniGPT-4, Gemma3, and mPLUG-Owl2 demonstrate consistent reductions in object hallucination on CHAIR, POPE, and MME while maintaining caption fluency and overall response quality. We further demonstrate the generality of our approach on the TempCompass video understanding benchmark and on discrete diffusion language models for grounded dialogue, showing that representation filtering reduces hallucinations even in temporal video reasoning and multi-step, sequence-wide denoising settings.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08167
PDF
-
SUMI: Scalable Unified Model for 3D Point Cloud Inference
2026-08-08 12:56:54Yanlong LI, Kanchana ThilakarathnaAbstract
Point cloud completion commonly follows a coarse-to-fine paradigm, where a low-density coarse shape is first predicted and then upsampled to the target resolution. Although recent methods have improved global structure recovery, the fine stage often remains limited by simple upsampling and insufficient interaction with coarse structural features, making local detail reconstruction challenging. We propose SUMI, a diffusion-enhanced refinement module for coarse-to-fine point cloud completion. Unlike prior diffusion-based completion methods that use diffusion as a standalone point generator, SUMI injects noisy geometric features into cross-attention with coarse structural features, enabling reverse denoising to refine local geometry while preserving global consistency. SUMI can also be integrated into existing coarse-to-fine models as a flexible refinement module. Experiments on PCN, ShapeNet-55/34, and MVP demonstrate consistent improvements over strong baselines. SUMI achieves the best overall CD and F1-score on PCN, reduces CD by up to 16.1% on ShapeNet-55, and obtains the best CD across all output densities on MVP.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08115
PDF
-
Archer: Adaptive Reuse of Cached Hidden States for Efficient Rollback in Diffusion Language Models
2026-08-08 12:13:10Xuning He, Zinan Sheng, Yongding Tao, Huanyu Liu, Ge Li, Xue Jiang, Yihong DongAbstract
Diffusion language models (DLMs) iteratively refine a sequence, allowing earlier predictions to be revised as context evolves. This rollback capability distinguishes them from irreversible autoregressive generation, but makes inference costly. Every denoising update alters the global context, forcing both prompt and response states to be recomputed even though only response tokens are revisable. Key-value (KV) caching could reduce this cost, yet conventional caching assumes immutable historical states and is therefore difficult to reconcile with this http URL this paper, we introduce Adaptive Reuse of Cached Hidden States for Efficient Rollback (Archer), a training-free KV caching method for rollback-capable DLMs. Archer asymmetrically keeps the mutable response synchronized with the current hypothesis while reusing prompt K/V within a bounded state neighborhood. Although prompt representations also change under bidirectional attention, their token identities remain fixed; bounded reuse therefore amortizes repeated prompt computation without caching mutable response states. It also delays feedback from tentative tokens, reducing premature reinforcement of transient high-confidence errors and giving rollback more opportunity to correct them. Our analysis characterizes prompt reuse as a reversibility-aligned cache boundary, bounds its state-dependent approximation error, and gives a decoder-margin condition for preserving full-refresh this http URL DLM acceleration often trades quality for speed. Archer shifts this frontier, attaining the best mean performance of 33.63% together with a 2.57x mean speedup on the main suite. Across evaluated settings, it improves Pass@1 by up to 3.05 points and reaches up to 2.95x speedup. Controlled analyses connect the quality gain to delayed prompt feedback and validate state-aware refresh. Our code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08086
PDF
-
Distilling Physical Priors into Streaming World Models
2026-08-08 07:35:10Liangliang Zhao, Junying Wang, Danni Yang, Yifan Chang, Bin Fu, Yu Qiao, Bowen Zhou, Yihao LiuAbstract
Streaming world models predict future visual states online while maintaining physically coherent dynamics over long horizons. However, their rollouts often violate basic physical constraints. A common approach distills pretrained bidirectional DiTs into few-step causal generators. However, this paradigm suffers from two fundamental limitations: generic bidirectional teachers acquire limited physical priors from visually oriented pretraining, and the limited priors suffer further loss during bidirectional-to-causal distillation. We present PhyS, a three-stage framework for distilling physical priors into streaming world models. To acquire physical priors from real-world interactions, we construct PhyS-120K, a dataset of 120K real-world physical-interaction videos spanning rigid-body dynamics, soft-body deformation, fluid phenomena, and phase transitions. Each video is annotated with structured descriptions of object properties and causal state transitions. Physics-aware supervised fine-tuning injects the physical priors into a bidirectional 14B DiT teacher, which we then distill into a lightweight 1.3B causal DiT for few-step autoregressive streaming generation. Finally, we use online reinforcement learning to incentivize the distilled model to generate physically plausible rollouts and further propose Temporal Credit Routing (TCR) to address temporal credit assignment. TCR evaluates physical consistency over overlapping temporal windows and routes the resulting group-relative advantages to temporally aligned denoising actions. On PhysicsIQ, PhyS improves the Wan2.1-14B teacher by 18.2\% and the Self Forcing, Rolling Forcing, and Causal Forcing by 23.7\%, 14.8\%, and 31.4\%, respectively. Results also improve the physics-aware video benchmarks VideoPhy, VideoPhy2, and PhyGenBench. The dataset, code, and more sample videos are available on our Project Page.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07981
PDF
-
UniScale: Arbitrary-Scale Industrial Anomaly Generation
2026-08-08 02:30:19Shilei Zeng, Linxin Guan, Xurui Li, Yaohan Tang, Yu ZhouAbstract
Industrial anomaly inspection faces a major challenge due to the lack of real-world anomaly samples. While generative models are used to create anomaly data, existing methods still struggle when handling small-scale this http URL failure occurs because extreme downsampling in diffusion models causes the information of small anomalies to be lost in the latent this http URL address this, we introduce UniScale, a unified training and inference framework for high-fidelity industrial anomaly generation across arbitrary this http URL training, we introduce an Error-Suppressed Multi-Scale Training (EMT) strategy, which enables the model to learn the rich location-aware textures of anomalies, while suppressing upsampling-induced interpolation errors in texture acquisition, ensuring the model is capable of learning small-scale anomalies, while remaining effective for regular scale this http URL inference, we propose Generation-then-Fusion Denoising. It decouples anomaly generation from background integration, preventing small anomalies from being this http URL experiments demonstrate that our method outperforms state-of-the-art competitors in both anomaly generation quality and downstream detection this http URL achieves a relative IS(a) improvement of 45.86\% (from 1.81 to 2.64) on VisA and 37.70\% (from 1.22 to 1.68) on MVTec AD 2, while also improving the downstream pixel-level IoU by 4.22\% on VisA and AUROC by 6.55\% on MVTec AD 2. Code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07864
PDF
-
Diffusion LLMs as Targets and Adversaries: Mechanistic Safety Exploits
2026-08-07 17:17:18Elena Dumitrescu, Gert Lek, Lydia Y. Chen, J\'er\'emie DecouchantAbstract
Diffusion Large Language Models (DLLMs) replace autoregressive next-token prediction with iterative parallel denoising, yet their internal safety mechanisms remain poorly understood. In this work, we investigate DLLMs both as targets and as adversaries, exposing mechanistic vulnerabilities in diffusion-based alignment. We first show that safety alignment in DLLMs remains sparse and transferable across architectures. DLLMs initialized from autoregressive predecessors inherit the same mechanistic safety footprint as their source models, enabling transfer attacks via direct safety neuron mapping and pruning. Self-pruning increases attack success rates (ASR) from 2.6% to 73.8% on LLaDA and from 1.9% to 86.6% on Dream, while transfer pruning from Qwen2.5 increases ASR from 1.9% to 73.2% on Dream and from 7.0% to 86.3% on Fast-dLLM. Building on these findings, we introduce SN-Guided Diffusion, a fully offline black-box jailbreak framework that steers the diffusion process away from safety-triggering regions using a weighted safety neuron loss, which achieves near-perfect prompt separability (AUROC = 1.0 for benign-vs-jailbreak discrimination). Across multiple open and proprietary targets, our method achieves a transfer ASR of up to 77.1% on Llama-3-8B-Instruct, 86.9% on Qwen2.5-7B-Instruct, and 74.3% against Gemini-2.5-Flash-Lite, while requiring only 20 generation episodes per prompt. Compared to prior jailbreaking frameworks, our method achieves competitive transferability with orders-of-magnitude lower generation cost. Our codebase is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07430
PDF
-
Omni-LiveAvatar: Minute-Level Real-Time Streaming Joint Audio-Visual Avatar Generation
2026-08-07 15:34:15Lunjie Zhu, Xingtong Ge, Fangyu Lin, Yi Zhang, Zhening Liu, Mengfei Li, Yumeng Zhang, Guanglu Song, Yu Liu, Jun ZhangAbstract
Joint audio-video generative models serve as foundation for immersive and interactive digital-human generation. Nevertheless, most existing models rely on bidirectional attention and multi-step denoising and can generate only short clips, making them unsuitable for real-time interaction over extended durations. We present Omni-LiveAvatar, the first framework for minute-level, real-time streaming joint audio-video avatar generation. Specifically, we propose (1) a progressive autoregressive distillation pipeline that transfers a large bidirectional joint audio-video diffusion model into a few-step autoregressive generator without auxiliary stabilization mechanisms; (2) a synchronized audio-video long-short-term memory that preserves global consistency under a bounded memory budget; and (3) a hierarchical rolling prompt planning strategy that enables coherent semantic evolution and seamless prompt transitions. Extensive experiments show that Omni-LiveAvatar generates high-quality, synchronized minute-level avatars in real time. In terms of speed, it achieves a 33$\times$ generation speedup over its teacher, LTX-2, on a single NVIDIA H200 GPU; in terms of generation quality, it outperforms accelerated baselines across visual quality, audio quality, cross-modal synchronization, and human fidelity. Our code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.13602
PDF
-
Fluid-DiT: Graph-Free Diffusion Transformers for Fluid Flow Simulations Learning
2026-08-07 12:28:45Shentong Mo, Guolin KeAbstract
Simulating complex fluid flows requires capturing full equilibrium distributions rather than just mean trajectories, yet high-fidelity solvers remain computationally prohibitive. Recent advances, such as Diffusion Graph Networks (DGNs), have combined diffusion models with graph neural networks to sample equilibrium states directly from unstructured meshes, enabling distributional accuracy even from short simulations. However, graph-based diffusion approaches suffer from hand-crafted architectural constraints, limited receptive fields in message passing, and costly multi-scale designs, which restrict scalability to larger and more complex domains. We propose Fluid-DiT, a Graph-Free Diffusion Transformer that replaces graph message passing with attention-based denoising, eliminating explicit graph design while preserving the ability to model distributions of chaotic flows. Our framework introduces a latent-space formulation that disentangles geometric fidelity from distributional learning, reducing high-frequency artifacts and accelerating sampling. By leveraging the transformer's global receptive field, Fluid-DiT naturally captures both local flow structures and long-range correlations without requiring hierarchical graph coarsening. On canonical benchmarks including laminar cylinder wakes, ellipse-flow systems, and turbulent 3D wing experiments, Fluid-DiT consistently outperforms graph-based diffusion baselines in both sample quality and distributional accuracy, achieving higher $R^2$ correlations and lower Wasserstein distances. Moreover, it generalizes robustly from short, incomplete trajectories to unseen Reynolds numbers and geometries, demonstrating strong scalability.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07161
PDF
-
Selective Cross-View Consistency for World Action Models: Held-Out Viewpoint Robustness Without Test-Time Camera Information
2026-08-07 12:03:23Bingqi Huang, Bingchuan Wei, Yingkai Cai, Zhaokui WangAbstract
World action models (WAMs) jointly denoise future video frames and robot actions, and the video prior is expected to generalize their control. Camera viewpoint change remains one of their hardest perturbation axes. We study a question specific to this model class: when training with same-state cross-view image pairs, on which output coordinates should a consistency loss be imposed? The WAM denoising target mixes view-covariant coordinates, namely the predicted future scene, with view-invariant coordinates, namely the action chunk, future proprioception, and value. We show that consistency applied to the covariant block is provably harmful, shrinking legitimate view-specific content to a fraction $1/(1+4\lambda)$ of its true value, and we verify this shrinkage law in controlled experiments. Selective cross-view consistency (SCVC) therefore constrains only the invariant block, requires no camera labels, extrinsics, depth, or view synthesis at training or test time, and leaves the deployment interface unchanged. We introduce a carve-and-hold-out evaluation protocol on the LIBERO-Plus camera track that separates a distribution-matched ceiling from genuine interpolation and extrapolation to held-out viewpoints, with a matched pair-trained control isolating the effect of the consistency term from pair exposure. On held-out orbital viewpoints beyond the training envelope, SCVC improves closed-loop success over the matched control by 12.2 points (95% CI [7.4, 17.0]; +15.5, CI [11.7, 19.4], under an independent second seed) -- an effect two further camera axes replicate -- while interpolation within the envelope shows no gain in either seed (-1.2 and -4.3 points) and in-distribution competence is preserved (-0.6, -0.2). We also report a cross-backbone audit showing that published camera-robustness numbers are confounded by wrist-camera pose stability.
Abstract (translated)
URL
https://arxiv.org/abs/2608.21402
PDF
-
Structural Guidance for Unified Joint Demosaicing and Denoising
2026-08-07 10:14:20Qixin Zheng, Ping Chen, Qiangqiang Shen, Haijin ZengAbstract
Joint demosaicing and denoising is a fundamental step in camera image signal processing, yet remains challenging because different Bayer-like color filter arrays (CFAs) and sensor noise jointly corrupt both color sampling and image content. Existing unified restoration networks explicitly model CFA geometry but are still driven primarily by pixel-level supervision, making them prone to structural degradation around edges, repetitive textures, and moiré patterns where local evidence is unreliable. We attribute this limitation partly to the absence of explicit structural guidance beyond pixel-level reconstruction supervision. Motivated by this observation, we propose a structural-guided unified restoration framework that injects pretrained structural knowledge into CFA-aware image restoration. Our model receives a unified five-channel observation consisting of the raw mosaic, CFA masks, and a noise-level map. A SwinIR restoration branch reconstructs pixel details under CFA-conditioned modulation, while a parallel structural reasoning branch extracts complementary structural cues from a sparse pseudo-RGB observation. To bridge the substantial domain gap between sparse noisy sensor data and the natural-image pretraining domain of the structural encoder, we introduce a lightweight trainable adapter before residually fusing structural and restoration features. A shared decoder jointly predicts the restored RGB image and an auxiliary clean mosaic, providing supervision in both image and sensor domains. Extensive experiments across multiple CFA patterns and noise levels demonstrate consistent improvements over state-of-the-art unified and CFA-specific methods, indicating that adapted structural priors can enhance robust camera image restoration. The source codes and dataset are provided in the supplementary material.
Abstract (translated)
URL
https://arxiv.org/abs/2608.09995
PDF
