Denoising
-
LiveLight: Real-time Streaming Video Relighting with Interactive Control
2026-08-03 06:45:00Yue Ma, Jiangming Wang, Yucheng Wang, Xilai Wang, Zhiyuan Li, Xinyu Wang, Hongyu Liu, Ruofan Liang, Songchun Zhang, Yuxuan Xue, Qifeng ChenAbstract
We present LiveLight, the first diffusion-based framework for real-time streaming video relighting with interactive 3D lighting control. Achieving this is non-trivial, as it requires overcoming three critical challenges: effectively injecting dynamic 3D lighting into a diffusion model, maintaining high-fidelity generation under an extremely low NFE (Number of Function Evaluations) budget for real-time speed, and facilitating continuous streaming for interactive control. To address these pain points, we propose three key designs. First, for accurate lighting injection, we propose a lightweight adapter that feeds Multi-Plane Light Irradiance (MPLI) conditions-depth-aware irradiance maps encoding 3D lighting geometry-directly into the diffusion backbone. Second, to prevent rendering quality degradation at low NFEs towards real-time distillation, we introduce a geometry-guided feedback branch. This training-time constraint leverages a frozen geometry estimator to enforce depth- and normal-consistent relighting, ensuring geometrically plausible shading without adding inference overhead. Finally, to enable streaming interaction, we develop a progressive rolling-window strategy that maintains a denoising ladder of latent chunks at varying noise levels. By propagating intermediate states, this strategy guarantees temporal coherence and supports arbitrarily long video relighting with per-frame reference refresh. Extensive experiments on real-world and synthetic benchmarks demonstrate that LiveLight achieves state-of-the-art relighting quality while running at real-time speed, significantly outperforming offline baselines in temporal stability, lighting controllability, and user preference. To foster real-time interactive relighting research, we will publicly release our models, training data, and synthetic data generator.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01771
PDF
-
Entity-Aware Sequence Transduction for Player-Centric Ball Action Spotting
2026-08-03 04:48:54Ruifeng Wang, Di Yang, Jiangtao WangAbstract
Player-centric ball action spotting requires temporally precise event detection together with actor attribution in crowded, partially observed multi-agent sports videos. Existing Denoising Sequence Transduction (DST) baselines treat the player-role dimension as part of a flattened frame-level representation, which weakens the inductive bias for modeling player-specific temporal evolution and inter-player interactions. To address this limitation, we propose Multi-Entity Denoising Sequence Transduction (ME-DST). ME-DST keeps the role-slot dimension throughout encoding. It uses temporal attention to model the history of each role slot, and spatial attention to exchange information across role slots at each frame. This factorized design gives the model a direct structure for separating within-player evolution from inter-player context. We also add learnable role embeddings, tracking-derived tactical features, and fused visual predictions from X3D-L and Swin3D-S. Experiments on the FOOTPASS dataset show that ME-DST reaches a Micro F1 of 0.778. This improves the strongest official TAAD+DST baseline by 10.3 percentage points. Controlled ablations show that preserving the entity axis and encoding role identity are central to this gain. These results suggest that explicit entity modeling is an effective inductive bias for player-centric sports event understanding.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01696
PDF
-
UDT: Reconciling U-Nets and Diffusion Transformers with Data-Adaptive Token Reduction
2026-08-02 15:07:21Junno Yun, Ya\c{s}ar Utku Al\c{c}alar, Mehmet Ak\c{c}akayaAbstract
Diffusion Transformers (DiTs) have emerged as a core architecture in generative modeling due to their scalability and adaptability to multimodal tasks. DiTs comprise isotropic transformer blocks, and learn representations progressively across depth, where the denoising objective drives later layers to focus on fine-detail reconstruction. This results in degraded representation quality and an imbalanced encoder-decoder behavior. Prior approaches such as representation alignment (REPA) mitigate this by encouraging stronger early representations via training regularization. Alternatively, U-Net-style DiT architectures introduce explicit multi-scale encoder-decoder structures for improved convergence. But they build on standard U-Net wisdom via learnable operators for spatial downsampling, which are not well-suited to transformer architectures, introducing inefficiencies and compatibility issues with components such as cross-attention and representation regularization. In this work, we propose UDT, a U-Net diffusion transformer that combines the representation power of DiTs with the encoding-decoding benefits of U-Nets, through data-adaptive token merging for downsampling and upsampling, while preserving the DiT token dimension. Our baseline UDT architecture outperforms existing U-Net DiTs and achieves performance comparable to REPA across all model sizes. Furthermore, using architectural optimization and REPA, UDT outperforms SiT's 7.9 FID at 1400 epochs (w/o CFG) within 40 epochs (~ 40x faster convergence) for XL model size on 256x256 ImageNet. Finally, it achieves strong image generation performance with CFG, reaching FID of 1.38 (320 epochs) with SD-VAE and 1.35 (500 epochs) with VA-VAE, providing a new backbone for DiTs with strong empirical benefits.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01298
PDF
-
TurboClear: One-Step Object-Effect Removal via Region-Calibrated Distribution Matching and Fusion
2026-08-02 14:56:13Jiawei Guo, Junxian Li, Yixin Tang, Bingya Zhang, Jiaxin Lu, Yulun Zhang, Shangchen ZhouAbstract
Recently, diffusion-based removal methods have achieved promising visual quality in removing both target objects and their associated effects. However, they typically rely on multi-step denoising, leading to high inference cost. Directly applying existing one-step distillation methods is also suboptimal, since their global objectives lack explicit region-wise calibration and may weaken the asymmetric edit-and-preserve behavior required by object-effect removal. To address these challenges, we propose TurboClear, a one-step SDXL-based object-effect removal model. During training, we design Region-Calibrated Distribution Matching (RDM) for region-aware distillation to preserve the teacher model's asymmetric edit-and-preserve behavior. Furthermore, we propose Learnable Spatial Fusion (LSF) for lightweight inference-time fusion. Extensive experiments show that TurboClear significantly improves inference efficiency while maintaining competitive visual quality. TurboClear reduces the computational overhead by up to $40.04\times$ compared to ObjectClear, and by up to $665\times$ against the Flux-based method OmniPaint, all while maintaining comparable or better visual removal quality. Code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01288
PDF
-
Astrolabe: Spherical-Map Guidance Across Diffusion Pipelines for Full-Body Capture from Unconstrained Images
2026-08-02 14:41:22Shuliang Zhu, Qi Wang, Ryugo Morita, Jinjia ZhouAbstract
Full-body capture from unconstrained photographs requires global correspondence across arbitrary views, poses, crops, and occlusions. Yet pose, geometry, and foundation features estimated in this setting are too unreliable for dense matching or appearance transfer, while diffusion rectifiers and optimization pipelines expose no common interface for consuming such uncertain correspondence. Our insight is that correspondence need not be locally accurate: its coarse viewpoint and body layout can still organize how a diffusion prior adapts and guides reconstruction. We introduce \emph{Astrolabe}, a host-portable adapter built on frozen viewpoint-guided spherical maps (SPH). A fixed bounded transform converts SPH into a spatial noise shift, which is matched during prior adaptation and reused during downstream denoising or score-distillation guidance in both pipeline categories. When a rectifier exposes a reference router, the same target/reference SPH additionally supplies coarse compatibility scores to select native appearance features; router-free optimization uses only the shared shift path. Astrolabe therefore follows one SPH--shift--adapt--guide process without dense warping or a learned control branch. Across Puzzle-IOI and 4D-Dress, it improves all reported image metrics in both hosts and all paired Puzzle-IOI geometry metrics; image gains extend to rear views, while 4D-Dress geometry remains stable overall.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01276
PDF
-
DeltaFlow: Noise-Adaptive Bidirectional Gated Delta Networks for Embedded Language Flows
2026-08-02 13:48:40Guangfu Guo, Xiaoqian Lu, Linsey Pang, Weiran Yao, Haolin Chen, Kunpeng Liu, Long ChengAbstract
Embedded Language Flows (ELF) rely primarily on full non-causal attention for iterative denoising, repeatedly incurring quadratic sequence-mixing cost at each sampling step. Gated Delta Networks (GDNs) provide an efficient recurrent alternative, but their standard causal formulation cannot directly capture the bidirectional context required by ELF. We introduce DeltaFlow, a noise-adaptive bidirectional GDN backbone for continuous language denoising. We study two variants: DeltaFlow-A, which alternates scan directions across layers, and DeltaFlow-P, which performs parallel forward and backward scans within each layer. We further introduce noise-adaptive memory control and scheduled Temporal State Consistency (TSC) to stabilize hidden representations across nearby noise levels. On OpenWebText, using a 32-step stochastic differential equation sampler, DeltaFlow-P reduces generated perplexity from 24.218 for the full-attention ELF baseline to 21.228 while maintaining comparable unigram entropy, with 36B training-token exposure compared with 45B for the baseline. In a denoiser-only benchmark, DeltaFlow-P achieves a 2.72x throughput speedup over the full-attention baseline at a sequence length of 16k. These results show that DeltaFlow is a promising alternative to dense attention for efficient continuous language denoising.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01240
PDF
-
EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation
2026-08-02 13:11:22Jinsong Lin, Zikang Pan, Wanhao Liu, Chi Kit Ng, Liangjing Shao, Zihang Yu, Ziyu Wang, Yin Wang, Jiaxi Wang, Jeremy Yuen-Chun Teoh, Zhiyong Xiong, Huxin Gao, Hongliang RenAbstract
Autonomous endoscopic navigation can reduce clinicians' operational burden, yet robust control remains challenging due to tissue deformation, transient occlusions, and rapidly changing viewpoints. Existing learning-based policies typically predict actions from current observations without explicitly modeling future dynamics, limiting their robustness and reliability in safety-critical settings. World Action Models (WAMs) offer a promising alternative by coupling predictive visual dynamics with action generation, but extending them to robotic endoscopy remains challenging due to limited training data, restricted viewpoint diversity, deformable anatomy, and high inference latency. We present EndoWAM, which is, to our knowledge, the first WAM for generalizable robotic endoscopic navigation. EndoWAM introduces future grounding, which predicts task-relevant target regions in future observations from intermediate denoising features of a video world model. Specifically, EndoWAM couples a lightweight diffusion transformer for future target-region prediction with a discrete action expert through a shared predictive representation. This design injects target-aware supervision into predictive dynamics modeling, improving robustness to visual degradation and viewpoint changes while enabling real-time control in a single denoising pass. We further introduce EndoMotion, a robotic endoscopic motion dataset spanning three anatomically distinct procedures: ureteroscopy, esophagoscopy, and endoscopic retrograde cholangiopancreatography (ERCP). EndoWAM consistently outperforms all baselines and alternative grounding strategies, while demonstrating strong zero-shot generalization to unseen viewpoints, environments, and targets. These results establish EndoWAM as a predictive, target-grounded framework for accurate, generalizable, and long-horizon navigation in visually constrained endoscopic environments.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01221
PDF
-
MiniWorld: Democratizing the Training of Video World Models from Scratch
2026-08-02 09:57:02Yian Zhao, Ruochong Zheng, Hongcan Guo, Yu Yan, Jian Zhang, Jie ChenAbstract
Video world models predict future observations conditioned on historical observations and control signals, enabling long-horizon generation through autoregressive state transitions. Unlike conventional video generation models that primarily capture visual appearance and motion, video world models learn the underlying dynamics governing environment evolution under agent actions, providing a foundation for embodied AI and interactive simulation. Recent progress has largely relied on adapting pretrained video generation models through post-training or distillation. Although effective, these approaches often require complex training pipelines, substantial computational resources, and suffer from the mismatch between bidirectional pretraining and causal streaming inference. Recent studies have shown that training autoregressive video world models from scratch is feasible and scalable. However, the community still lacks a lightweight, transparent, and fully reproducible baseline trainable end-to-end with modest computational resources. We present MiniWorld, a reproducible framework for training streaming video world models from scratch. MiniWorld employs a block-causal Video Diffusion Transformer trained with Flow Matching in the latent space of a pretrained Video VAE. Building on Diffusion Forcing, it adopts a chunk-wise non-decreasing noise schedule and two-stage continued training to improve temporal modeling and stability. During inference, MiniWorld combines a rolling KV cache with pipelined asynchronous denoising for efficient streaming generation under bounded computation. The entire model can be trained within several days on a single 8-GPU server. By releasing the training and inference codebase and pretrained checkpoints, we hope MiniWorld will facilitate future research on video world modeling.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01127
PDF
-
Lethe: How Hard Is It to Forget? A Benchmark for Federated Unlearning in Medical Imaging
2026-08-02 08:50:20Shengchao Chen, Ting ShuAbstract
Federated learning enables medical-imaging models to be trained across hospitals, and privacy law, most explicitly the GDPR ``right to be forgotten'', turns removing a hospital's, a class's, or a patient's influence from such a model into a federated unlearning problem. This need is most acute in medicine, where patients withdraw consent and hospitals leave collaborations. Yet nearly all unlearning evidence comes from natural images, whose heterogeneity and task structure differ sharply from clinical data, so it is unclear whether existing methods transfer, and no shared protocol covers clinical data. We present Lethe, a benchmark for federated unlearning in medical imaging. It evaluates twelve methods across eight task families, from classification and segmentation to denoising, cross-modality synthesis, and vision-language question answering, at three forgetting granularities and against a retrained gold standard on utility, privacy, and cost. The central result is that what separates methods is the difficulty of the forgetting request, not the method itself. The easy removals that dominate the literature leave the methods that preserve utility indistinguishable, while only hard ones separate them. More striking, on the many medical tasks that generalize across sites, forgetting a client barely changes task performance, leaving residual membership as the signal that must be erased.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01094
PDF
-
Diffusion-Based Body Schema Learning Enabling Abnormal-State Adaptation in Musculoskeletal Robots
2026-08-02 06:10:03Kento Kawaharazuka, Shuhei IkemotoAbstract
Musculoskeletal robots require an internal body schema that remains consistent under a wide range of physical state changes, including abnormalities such as muscle rupture and actuator jamming. Conventional approaches based on autoencoders or variational autoencoders learn average behaviors by projecting sensor and actuator signals into a low-dimensional latent space; however, exploration within the latent space alone has limited capability to handle out-of-distribution or abnormal states that are not included in the training data. To address this limitation, this study proposes a diffusion-based framework for body schema learning in musculoskeletal robots. Unlike generative models that operate through low-dimensional latent spaces, diffusion models can directly and iteratively estimate physically consistent sensor and actuator values in the high-dimensional space through a denoising process, even under partial observations and constraints, without requiring retraining. By formulating body schema adaptation as a gradient-guided denoising process, the proposed method enables adaptive estimation of appropriate muscle lengths and muscle tensions even under abnormal conditions such as muscle rupture and actuator jamming. The validity of the proposed framework is verified through simulation experiments using a musculoskeletal robot model.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01029
PDF
-
Where Does Generative Difficulty Reside? An Empirical Study of Target Representations
2026-08-01 12:32:31Marcel Plocher, Bernhard Sch\"olkopf, Andreas Geiger, Gege GaoAbstract
The target representation defines the distribution an image generator must learn, yet it is often treated as an interchangeable interface. This assumption is particularly questionable for continuous masked generators, which combine contextual inference from visible tokens with conditional modeling of each missing token. We study raw pixels, SD-VAE latents and DINOv2 as well as MAE representation-autoencoder features within a unified masked autoregressive rectified-flow model. Under a shared ImageNet training budget, these spaces exhibit distinct optimization and inference regimes. DINOv2 converges fastest in both iterations and computation but benefits strongly from a wider local denoiser and direct context fusion. Pixels optimize substantially more slowly and require a different prediction, masking, and guidance configuration. MAE reconstructs images more faithfully and exhibits clear semantic clustering, yet produces generations substantially worse than DINOv2. The representations also respond differently to classifier-free guidance and occupy distinct precision-recall trade-offs. Together, our results show that compression, reconstruction fidelity, token dimensionality, and visible semantic clustering do not individually predict generative behavior. Instead, target representations redistribute difficulty across contextual modeling, per-token denoising, and inference-time distributional control.
Abstract (translated)
URL
https://arxiv.org/abs/2608.00626
PDF
-
Practical Noise Modeling for SPAD Intensity Imaging
2026-08-01 07:25:41Wendi Liu, Yujie Lu, Zengxi Zhang, Haiyang Jiang, Weihang Ran, Yinqiang ZhengAbstract
Single-photon avalanche diode (SPAD) cameras are promising for low-light and high-dynamic-range intensity imaging, but their practical use is limited by complex sensor-specific noise. Unlike time-correlated single-photon counting (TCSPC) systems, SPAD cameras record whether at least one detection occurred in each gate without photon timestamps in intensity imaging mode, making explicit noise decomposition difficult. We present a practical noise modeling and calibration framework for SPAD intensity denoising. Our forward model describes binary-frame accumulation with a Binomial observation process, models signal-independent dark noise as an exposure-dependent pure dark count term plus an exposure-independent dark-frame bias term, and incorporates pixel-wise response non-uniformity. We design a dedicated calibration procedure for the proposed model and use it to build a count-domain noise-synthesis pipeline for network training. For denoising, we further design a SPAD-specific dark-shading correction (SPAD-DSC) to remove most systematic noise before network training. We construct a real-world SPAD intensity dataset for testing. Experimental results demonstrate the superiority of the proposed noise model.
Abstract (translated)
URL
https://arxiv.org/abs/2608.00489
PDF
-
DreamTraj: Generating 6-DoF Object Trajectories by Reading Unrendered Video Diffusion Latents
2026-08-01 07:24:41Tongsheng Ding, Zhen Luo, Yixuan Yang, Boyu Wang, Luyang Xie, Jinyu Yang, Feng ZhengAbstract
Accurate prediction of object trajectories during manipulation is essential for closing the perception-action loop. Progress is limited on two fronts: available datasets lack fine-grained language-to-motion annotations, and existing predictors either rely on privileged inputs such as video, depth, or CAD models, or recover motion from fully generated videos through costly, error-prone perception pipelines. We close the supervision gap with the MOVE dataset, 5,038 object-centric egocentric trajectories, each paired with a fine-grained natural-language instruction rather than a coarse verb-noun label. We further propose DreamTraj, which predicts a 6-DoF object trajectory from a single RGB image and a task instruction, requiring no video, depth, or CAD model at inference: rather than generating a video, it reads motion from the internal representations of a frozen image-to-video diffusion model at an early denoising step. A lightweight flow-matching Reader decodes query-key attention tracks and pooled hidden states into relative 6-DoF poses. To our knowledge, this is the first approach to directly decode object 6-DoF trajectories from intermediate video diffusion representations rather than generated pixels. DreamTraj sets a new state of the art on both translation and rotation against forecasters that consume multi-frame or privileged inputs, and runs 4.6x faster than generate-then-extract pipelines.
Abstract (translated)
URL
https://arxiv.org/abs/2608.00486
PDF
-
DiffusionGemma Technical Report
2026-07-31 16:11:46DiffusionGemma Team, Adrien Ali Ta\"iga, James Assiene, Daniele Calandriello, Rahma Chaabouni, Jo\~ao Gante, Tamara von Glehn, Nate Keating, Chris Knutsen, Martin Kukla, Tianlin Liu, Ivan Lobov, Ofir Nabati, Jo\~ao Gabriel Oliveira, Nicolas Perez-Nieves, Nastasia Prutianova, Bobak Shahriari, Jean Tarbouriech, Pavel Tyletski, \c{C}a\u{g}lar \"Unl\"u, Cindy Wu, Glenn Cameron, Jerome Connor, Sertan Girgin, Maarten Grootendorst, Alon Levkovitch, Eliya Nachmani, Omar Sanseviero, Piotr Stanczyk, Quentin Berthet, Andrew Campbell, Cl\'ement Crepy, Valentin De Bortoli, Arnaud Doucet, Romuald Elie, Alexandre Galashov, Klaus Greff, Alexis Jacq, David Ruhe, Yu-Han Wu, Sebastian Flennerhag, Brendan O'Donoghue, George Scrivener, Shantanu ThakoorAbstract
We introduce DiffusionGemma, an experimental open-weight language model that uses discrete diffusion to generate text at exceptionally high speed. Rather than decoding one token at a time, DiffusionGemma iteratively refines blocks of 256 tokens in parallel, avoiding the sequential decoding bottleneck of conventional autoregressive (AR) large language models. Instead of training from scratch, we obtain DiffusionGemma by fine-tuning the mixture-of-experts Gemma 4 model with 3.8B activated and 25.2B total parameters. Our compute-efficient two-stage training pipeline uses fewer than 10% of the starting AR model's total training token budget. The first stage uses supervised fine-tuning to teach bidirectional denoising, while the second stage combines reinforcement learning with sampler distillation to jointly improve generation quality and inference efficiency. DiffusionGemma establishes a new Pareto frontier for the trade-off between generation speed and model capability. Averaged across our full evaluation suite, it generates around 20 tokens per forward pass and achieves roughly 1,500 output tokens per second on a single NVIDIA H100 GPU, which is substantially faster than AR models even with state-of-the-art speculative decoding. DiffusionGemma also retains the starting model's support for thinking mode, multimodal inputs, and long contexts. Despite diffusion fine-tuning, it remains capable of AR generation with only minor performance degradation, suggesting a path toward hybrid diffusion-AR decoding.
Abstract (translated)
URL
https://arxiv.org/abs/2608.00146
PDF
-
Safe Vision Language Action Models via Barrier Enhanced Flow Matching
2026-07-31 15:56:19Kasra Sinaei, Hung-Chieh Wu, Donald EbeigbeAbstract
This article presents a modular inference framework that integrates Flow Matching generative models with formal Control Barrier Function (CBF) safety guarantees. Unlike existing methods that apply external safety filters to a model's final output, our approach modifies the Flow Matching denoising process within the model to inherently generate safe trajectories. By employing a smooth Log-Sum-Exponential aggregate barrier, we enforce safety over entire action chunks. This aggregate barrier ensures a minimal increase in computational overhead and does not alter the semantic intent of the model. We show that, within the proposed framework, the 2-Wasserstein distance between the generated distribution and the target distribution remains bounded. Our method eliminates the need for safety-specific datasets or costly model retraining, providing a versatile solution for safe inference. We validate the approach on two robotic manipulation platforms and a 2D navigation benchmark, verifying that our framework achieves reliable safety without degrading the success rate of the model.
Abstract (translated)
URL
https://arxiv.org/abs/2607.29569
PDF
-
MBDiff: Multi-view Behavior-aware Diffusion Model for Probabilistic Utility Data Imputation
2026-07-31 08:58:54Rongchao Xu, Lin Jiang, Dahai Yu, Ximiao Li, Guang WangAbstract
Utility data (e.g., electricity, water, and gas consumption), collected by ubiquitous sensors and embedded devices, often contains substantial missing values due to various factors such as device failures and data transmission issues. The data missingness can severely impact utility billing accuracy, hinder demand forecasting, and disrupt efficient utility supply management. As a result, utility data imputation has attracted much interest from both industry and academia. While many studies have attempted to address this issue, most of them rely on aggregated datasets for training, overlooking rich user behavior information, which could provide valuable insights for more accurate imputation. However, learning comprehensive user behavior from long-term, diverse, and incomplete utility data remains a significant challenge. Moreover, leveraging user behavior information to guide imputation is nontrivial due to the indirect nature of the correlations. To address these challenges, we propose MBDiff, a Multi-view Behavior-aware Diffusion Model for Probabilistic Utility Data Imputation. MBDiff incorporates two key technical components: (i) a multi-view User Behavior Extraction module that learns comprehensive user behavior from multiple perspectives, including global, local, and instance-level views; and (ii) a behavior-aware conditional diffusion model consisting of a reference selection module and a conditional attentional denoising network to impute utility data in a computationally efficient manner. We implement and evaluate MBDiff by collaborating with one of the largest municipal utility providers in Florida. Experimental results demonstrate our proposed MBDiff effectively outperforms state-of-the-art baselines, e.g., it improves 7.04% and 29.1% on the electricity and water usage datasets for block missingness imputation, respectively.
Abstract (translated)
URL
https://arxiv.org/abs/2607.29177
PDF
-
CORA-Diff: Confidence-Oriented Residual Acceptance for Efficient Diffusion Language Model Inference
2026-07-31 07:32:10Yifan Wu, Yufeng Zhang, Kenli LiAbstract
Diffusion language models (DLMs) update many tokens in parallel, yet practical decoders often use a fixed denoising horizon. Many predictions stabilize early, but blockwise decoding continues until all positions are resolved, causing repeated dense forward passes. Existing accelerators often rely on learned filters, modified scores, dependency models, or cache-specific mechanisms. We ask whether native trajectory signals can identify residual positions likely to match the deterministic dense endpoint. We propose CORA-Diff, a training-free method that preserves the original transfer rule and applies confidence-and-persistence gating only to positions that rule leaves unresolved. Accepted tokens remain visible as context, and the block terminates once all positions are resolved. This requires no backbone change, learned acceptance model, or logit modification. Our theory explains why high-confidence, persistent predictions are more likely to match the fixed-horizon dense endpoint, and paired post-intervention trajectories provide direct empirical support. We select one operating point on a separate GSM8K calibration subset and freeze it for all evaluations. Under a matched Learn2PD-style LLaDA protocol, CORA-Diff has the lowest measured runtime in all eight task-length settings. Task scores match or exceed dense decoding in five settings, and the largest observed drop is 1.22 points. Its incremental speedups over EOS-aware dense decoding are 2.70x and 3.32x on GSM8K and HumanEval. It also reaches 13.14x under the fixed-horizon 1024/1024 mechanism-isolation protocol and transfers to Dream without retuning at 3.18x-3.53x. These results show that native confidence and persistence enable reliable residual acceptance, reducing repeated denoising computation while preserving task quality.
Abstract (translated)
URL
https://arxiv.org/abs/2608.11235
PDF
-
Logographic Character Visual Pretraining via Semantic-based Contrastive Learning
2026-07-30 17:12:18Daqian Shi, Wei Cao, Xiaoyu Zheng, Lida Shi, Xiaolei Diao, Cedric M JohnAbstract
Current deep learning-based character vision studies, e.g., text recognition, character image denoising, and historical text completion, are offering new solutions for learning, managing, and utilizing character resources. However, the performance of these studies peaks only with large and balanced datasets, which is a rarity with real-world character datasets, especially for logographic character languages, e.g., Chinese. The imbalance in data distribution of logographic characters is a common issue due to differences in character usage frequency and new characters being continuously created. In this paper, we propose a novel method for logographic character recognition, which introduces a multi-modal learning approach using visual semantics and contextual semantics of characters. A novel pre-training strategy is designed to enhance deep visual representations, especially for datasets suffering from issues of imbalanced and rare instances, by extracting the contextual semantics of each character from the corresponding language models. We conduct experiments across various datasets to evaluate our character recognition method and further validate the contrastive pre-training strategy by several downstream tasks. Experimental results demonstrate the superiority of our method compared to state-of-the-art methods.
Abstract (translated)
URL
https://arxiv.org/abs/2608.00096
PDF
-
CDAE: Enhancing Perturbation Robustness in Pretrained Language Models with Contrastive Denoising
2026-07-30 14:04:08Sina Heydari, Amirreza Abbasi, Mohsen Hooshmand, Majid RamezaniAbstract
Pre-trained language models have significantly improved sentence representation learning, yet their embedding remain sensitive to semantic preserving textual perturbations such as synonym substitution, masking and word dropout. This work proposes a lightweight Contrastive Denoising Autoencoder (CDAE) that refines pre-trained BERT embedding by jointly optimizing contrastive and reconstruction objective to learn perturbation-invariant representation. We evaluate the proposed framework using multiple perturbation strategies with varying strengths and compare it against the original BERT embeddings and SimCSE. Experimental results show that CDAE consistently preserves higher embedding similarity under perturbations, with the improvements becoming more pronounced as framework effectively enhances representation stability while preserving semantic information, highlighting perturbation-invariant learning as a promising direction for improving sentence embeddings. The source code is publicly available at: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2607.28236
PDF
-
Where and When to Commit: Candidate-Aware Decoding for Diffusion Language Models
2026-07-30 13:04:47Chia-Ming Lee, Ming-Ching Chang, Xin Li, Yu-Lun Liu, Chih-Chung HsuAbstract
Diffusion language models (DLMs) expose a provisional prediction at every denoising step, creating an opportunity for generation-time early exit that stops decoding before the schedule is exhausted. Existing early-exit gates decide termination from fixed-region confidence statistics or schedule-dependent rules, evidence too coarse for a decision that freezes every remaining position at once, so they fire prematurely on long chain-of-thought outputs whose answers stabilize only near the end. Adaptive sampling, the other axis of training-free acceleration, paces how quickly positions commit while decoding continues but never verifies that the output itself has stabilized. We introduce a training-free, candidate-aware early-exit framework that keeps the two axes separate and matches each decision to evidence of its own scope. Confidence-Verified Commit (CVC) governs when the sequence may stop by verifying confidence and sustained argmax stability over the dynamically extracted candidate span using a deterministic parser specified from each task's output format. Block-Wise Early Commit (BWEC) governs where to accelerate by applying a cheaper local rule to non-final blocks, while leaving the final block and global termination under CVC. We refer to their combination as LATCH (Localized Acceleration with Tracked-Candidate Halting). Unlike prior methods, LATCH needs no suffix-prompt construction; it is prompt-anchor-free but format-aware. We evaluate LATCH end to end on 11 tasks under zero-shot settings using LLaDA and Dream. LATCH stays within 2.0 percentage points of full-decoding accuracy across all 22 evaluation settings, with one frozen hyperparameter set that transfers cross-backbone untuned, while achieving end-to-end TPS speedups of 9.3-17.8x on short-answer tasks and 2.0-3.3x on long-reasoning tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2607.28166
PDF
