Restoration
-
RMR-Net: Degradation-Evidence-Guided Road-Image Restoration for Defect Detection
2026-08-09 23:29:35Amir Ghorbani, Amirali K. Gostar, WeiQin Chuah, Vahid Ghorbani, Aidan Blair, Alireza Bab-HadiasharAbstract
Vehicle-mounted road cameras are vulnerable to motion blur, defocus, poor illumination, and noise, which can erase thin cracks and pothole boundaries needed by road defect detectors. This paper presents RMR-Net, a compact task-aware restoration front end that estimates degradation evidence from the image, optionally fuses it with existing corruption context/parameters, conditions lightweight restoration blocks, and returns high-frequency pavement detail through a bounded residual path. The experimental scope is deliberately controlled: the conditioning information used on the Image and Vision Computing New Zealand (IVCNZ) pothole dataset and the Road Damage Dataset: Potholes, Cracks and Manholes (PCM) consists of saved synthetic-generator parameters, not measured vehicle telemetry. A clean-trained, frozen YOLO11s detector evaluates every image source. Across eight held-out degradation conditions, RMR-Net obtains the highest mAP50 in seven, including 0.140-0.427 for IVCNZ motion blur and 0.060-0.233 for PCM defocus. A compact ablation identifies the bounded detail path as the largest local contributor, while degradation conditioning and detector-aware stability terms provide complementary guidance.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08957
PDF
-
MRI super-resolution in ten sampling steps using a diffusion bridge model
2026-08-09 17:09:21Mojtaba Safari, Hang Yu, Zach Eidex, Mingzhe Hu, Ryan J. Sanford, Alexandru Florea, Shansong Wang, Chih-Wei Chang, Erik H Middlebrooks, Aditya Juloori, Stanley L. Liauw, Ralph Weichselbaum, Xiaofeng YangAbstract
Objective. MRI provides excellent soft-tissue contrast, but long acquisition times can cause patient discomfort and lead to motion artifacts, forcing a trade-off between spatial resolution and scan time. Diffusion-based super-resolution (SR) reconstructs high-resolution (HR) images from low-resolution (LR) inputs, but typically needs many sampling steps and initializes from a Gaussian prior ill-suited to image restoration. We developed an efficient diffusion framework that reconstructs HR MRI directly from LR data. Approach. We propose super-resolution diffusion bridge model (SR-DBM), a super-resolution diffusion bridge model that casts SR as a stochastic transport between the LR and HR image distributions. Through a Doob's h-transform of a mean-reverting stochastic differential equation, SR-DBM pins the process to the paired HR and LR images at its endpoints, initializing reconstruction from the measured anatomy rather than from Gaussian noise. The HR image is recovered by a deterministic reverse trajectory in which a network predicts the clean image at each of only ten sampling steps. We evaluated SR-DBM on ultra-high-field 7T brain T1 MP2RAGE maps and pelvic T2-weighted prostate images against nine comparison methods using PSNR, SSIM, GMSD, and LPIPS. Main results. SR-DBM attained the highest PSNR and SSIM and the lowest GMSD on both datasets (brain: 27.66+-1.52 dB, 0.96+-0.02, 7.96+-1.86$; prostate: 27.87+-2.29 dB, 0.80+-0.05, 8.38+- 1.44), with statistically significant gains over every comparison method (two-sided Wilcoxon signed-rank test with Holm correction, p<0.05). The strongest baseline, SR-EMamba, ranked second. Qualitatively, SR-DBM produced the smallest residual errors and best preserved fine structures and lesions.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08819
PDF
-
Degradation-Guided Underwater Image Restoration with Task-Oriented Latent Control
2026-08-09 12:08:34Xu Zhang, Xuhui Cao, Kangzhe Yuan, Laibin Chang, Yichu Xu, Shi Chen, Huan Zhang, Yong ChenAbstract
Degradation information in underwater images plays a dual role: its spatial and spectral cues can guide adaptive restoration, while degradation-entangled features may be propagated without explicit regulation during decoding. Existing methods largely overlook this dual role, either underexploiting degradation cues or directly forwarding encoder features through skip connections. To address this issue, we propose PROTEUS, which couples degradation-guided feature adaptation with task?oriented latent control. PROTEUS tackles this problem from two complementary perspectives. At the feature level, the Guided Dynamic Feature Modulation Block exploits spatially varying degradation cues to adapt feature processing across network stages. At the representation level, the task-oriented latent controller learns a structured control code under discriminative regularisation and uses it for channel-wise modulation of skip features, without requiring the code to form a metrically cleaner embedding. Extensive experiments on five paired and four non-reference underwater benchmarks demonstrate that PROTEUS achieves highly competitive restoration performance, with a favourable balance between restoration quality and computational cost.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08661
PDF
-
SDDBMs: Soft Denoising Diffusion Bridge Models
2026-08-09 09:19:46Shiyi Qi, Kun He, Mingmou LiuAbstract
Diffusion bridge models leverage Doob's \(h\)-transform to construct stochastic transports between arbitrary endpoint distributions, and have shown strong potential in image-to-image translation and restoration. However, most existing bridge models rely on hard endpoint conditioning, which forces the terminal state to match a prescribed target exactly. This hard constraint induces terminal-boundary singularities: the terminal law collapses to a Dirac measure, and the resulting drift coefficients become ill-conditioned near the endpoint. In this paper, we propose Soft Denoising Diffusion Bridge Models (SDDBMs), a generalized framework that regularizes diffusion bridges directly at the level of their terminal constraints. Instead of imposing an exact endpoint, SDDBMs prescribe a non-degenerate Gaussian terminal marginal under the transformed path measure, with a flexible terminal center and variance. Starting from this prescribed marginal, we develop a complete closed-form construction of the soft bridge, including the Gaussian terminal reweighting and soft \(h\)-function, the induced Gaussian forward marginals and \(\mathbf{x}_0\)-free dynamics. Theoretically, SDDBMs provide a unified probabilistic perspective that encompasses existing diffusion bridge models, including DDBMs, GOUB, and UniDB, as special cases under specific parameter choices. Extensive experiments on image restoration tasks demonstrate that SDDBMs achieve improved numerical stability and superior generation quality over existing bridge-based methods.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08594
PDF
-
MotionCraft: Latent World Modeling with Sparse Attention for Visual Upscaling
2026-08-09 07:54:43Rong Fu, Chunlei Meng, Yangchen Zeng, Xiaowen Ma, Yongtai Liu, Wangyu Wu, Shuo Yin, Zijian Zhang, Sicheng Li, Yingrui Ji, Chenhao Wang, Simon FongAbstract
Video super-resolution (VSR) aims to recover high-fidelity high-resolution videos from low-resolution inputs and is central to applications ranging from mobile capture to streaming and archival restoration. Existing approaches trade off among local-detail fidelity, long-range spatio-temporal modeling, perceptual realism, and efficiency: convolutional alignment techniques preserve local structure but suffer when motion is large or degradations are complex; transformer-based methods capture long-range dependencies yet require architectural or algorithmic adaptations to remain computationally feasible; and recent latent or diffusion-based generators synthesize rich texture but require specialized temporal constraints to maintain coherence. We present MotionCraft, a controllable VSR framework that formulates restoration as motion-aware latent state prediction inspired by world models and integrates adaptive sparse attention with an explicit user-accessible control interface. MotionCraft combines robust motion fusion, a Latent World Transformer that balances locality and targeted non-local interactions, and a compact conditional decoder to deliver temporally consistent, high-quality reconstructions under streaming constraints. Empirical evaluations show that MotionCraft achieves strong reconstruction and perceptual performance while enabling predictable trade-offs between temporal smoothness and reconstruction fidelity.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08553
PDF
-
Agentic AI-powered flexible fiber-bundle endoscopy for high-resolution NIR-II fluorescence imaging in vivo
2026-08-09 01:43:29Yanzhao Shi, Yuanhua Liu, Sixin Xu, Wayne Jason Li, Yuyuan Chen, Danyang Xu, Zhisheng Wu, Hanze Yu, Ian Yu-Hong Wong, Simon Ying-Kit Law, Hongjie Dai, Liangqiong Qu, Feifei WangAbstract
Fiber-bundle endoscopy offers a compact and flexible route for clinical fluorescence imaging through natural human orifices, but since its first report in the 1950s, it has remained limited by low spatial resolution, honeycomb artifacts, and inter-core crosstalk. The crosstalk becomes more pronounced at near-infrared-II wavelengths (NIR-II, 1000-3000 nm), a spectral window that offers superior contrast, resolution, and tissue penetration depth for biomedical imaging. Here, we present an AI-powered flexible endoscopy platform that overcomes these constraints through optical-computational co-design: optimizing ultrathin fiber bundles to mitigate crosstalk-induced image blur and enable high-fidelity image transmission across the visible-to-NIR-II spectral range, and developing an Agent-Guided Mixture-of-Experts (GAME) pipeline for honeycomb-artifact removal and image restoration. GAME provides a single restoration entry point for diverse biomedical images acquired with our endoscope, spanning cell, mouse and human samples. It dynamically routes each input to suitable restoration experts via a vision-language model, facilitating image reconstruction with a fourfold resolution improvement beyond the NyquistShannon sampling limit. The utility of our endoscope is demonstrated through in vivo NIR-II imaging of anatomical structures in mice, as well as imaging of the digital micromirror device (DMD)-projected human gastric tube and lymphatic system, paving the way for future clinical translation.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08402
PDF
-
Retrieval-Augmented Generation-Based Color Restoration for Low-Light Image Enhancement
2026-08-08 16:12:12Li-Wei Lu, Shaou-Gang MiaouAbstract
Recent low-light image enhancement (LLIE) methods have driven brightness and structural fidelity close to that of normally-exposed images, yet their outputs still exhibit systematic color shifts such as greenish skies, yellowish faces, and warm-tinted white objects. We attribute this to end-to-end LLIE training coupling brightness, structure, and color within a single network, leaving the color channels weakly supervised. We recast color restoration as an independent sub-problem and decouple it from brightness enhancement, realizing it as a general-purpose post-processing module built on retrieval-augmented generation (RAG). Rather than relying solely on parametric color priors learned during training, the module dynamically retrieves a reference image from an external high-quality color knowledge base and injects its color distribution into a color-restoration network to correct residual bias. The design has three components: (i) a dual-index FAISS retriever built on intermediate VGG19 features, capturing textural and structural similarity through global mean and variance statistics; (ii) GlobalSPHistAdaIN, which reduces the reference spatial-preserving color histogram to a global color vector and modulates network features via adaptive instance normalization, removing dependence on pixel-level correspondence; and (iii) a residual formulation that predicts a color correction over the front-end output. Across LOLv1, LOLv2-Real, and LOLv2-Synthetic, the module consistently improves color-specific metrics, and it remains effective when the front end is swapped among CPGA-Net++, LLFormer, FLIGHTNet, and IAT, confirming cross-front-end generality. Ablations show that a VGG19 dual index outperforms CLIP-based retrieval, indicating that color restoration depends on textural and structural similarity rather than high-level semantics.
Abstract (translated)
URL
https://arxiv.org/abs/2608.08211
PDF
-
IRPol-Fuse: Energy-structure coordination for infrared polarization fusion under low visibility
2026-08-08 01:35:25Zhuangfan Huang, Chusheng Fang, Xiaosong Li, Yang Liua, Xiaoqi Cheng, Haishu TanAbstract
Robust perception under low-visibility conditions requires fused imagery that jointly preserves infrared thermal saliency and polarization-derived structural details. However, existing infrared-polarization image fusion (IPIF) methods often overemphasize dominant infrared responses, causing weak yet informative polarization textures in dark regions to be suppressed. To address this issue, we propose IRPol-Fuse, an energy-structure coordinated IPIF framework for challenging low-visibility scenarios. The proposed framework contains three key modules: Polarization Attention Fusion for adaptive infrared-polarization allocation, Infrared Highlight Injector for highlight-guided infrared preservation, and Polarization Texture Injector for polarization texture restoration and fine-detail recovery. We further construct LI-PI, a dedicated infrared-polarization evaluation dataset for low-visibility and visually concealed scenes. Experiments on LI-PI and the public LDDRS dataset demonstrate that IRPol-Fuse achieves favorable performance in thermal target preservation, structural detail recovery, and visual naturalness. Region-aware evaluation and downstream object detection further verify that the proposed energy-structure coordination strategy effectively preserves both infrared target saliency and polarization-derived structural information. Code is available at this https URL .
Abstract (translated)
URL
https://arxiv.org/abs/2608.07848
PDF
-
Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination
2026-08-07 15:37:03Ruijie Hou, Yueyang Jiao, Zhao Wang, Yingming LiAbstract
Test data from public benchmarks inevitably leaks into pretraining corpora, inflating evaluation scores once memorized. \textbf{Contamination mitigation evaluation} intervenes in the decoding process to suppress memorization and restore a contaminated model's genuine capability, but its prevailing metric, the \textbf{G-AP} (\textbf{G}ap of \textbf{A}ggregate \textbf{P}erformance), is flawed. Discrete correct/incorrect readouts cannot characterize per-question performance, averaging before differencing lets over- and under-suppression cancel out, and uniform per-question weighting invites strategies to push solve probabilities onto the clean model's high-frequency values. We propose \textbf{SA-PPG} (\textbf{S}tratified \textbf{A}ggregate of \textbf{P}er-question \textbf{P}robability \textbf{G}aps): estimate each question's solve probability by sampling, difference it against the clean model per question, and aggregate within groups defined by the clean model's solve probability. Existing mitigation strategies first estimate where contamination lies and then operate on the estimate, so they are only as correct as the estimate. \textbf{RailCap} instead judges contamination during generation: whenever a sample falls back onto the greedy trajectory, the next trajectory token is capped to the runner-up, accumulating suppression until the response distribution becomes sufficiently dispersed. Across multiple contaminated models and benchmarks, SA-PPG reveals that prior strategies' restoration is substantially overestimated, while RailCap attains the lowest SA-PPG.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07341
PDF
-
Stoicheia: Character-Level Masked Diffusion for Ancient Greek Textual Restoration, Parsing, and Metrical Scansion
2026-08-07 14:07:43Eric Cullhed, Albin Th\"orn ClelandAbstract
We introduce Stoicheia, a 405M-parameter character-level masked-diffusion encoder for Ancient Greek whose input factors into five aligned, independently maskable planes: letters, word and sentence boundaries, diacritics, capitalization, and punctuation. A single backbone can therefore restore lacunae, re-segment, accentuate, and punctuate unspaced text without task-specific retokenization. We pretrain it on an open, revision-pinned corpus of 380M words and release eleven checkpoints: ten rotated, decontaminated folds, guaranteeing that for any given literary passage at least one released model has never seen its text, and one with no exposure to documentary texts. Three experiments - reconstruction of damaged inscriptions and papyri, morphosyntactic tagging and dependency parsing, and macronization with metrical scansion - each carry a matched random-initialization control, isolating what character-level diffusion pretraining contributes: 5.6 CER points on inscription reconstruction, 12.9 LAS on parsing, and 6.0 points of balanced accuracy on macronization. On Ithaca's own test split, with identical frozen samples and strict scoring, Stoicheia reduces character error relative to both prior state-of-the-art systems, from 24.6 (Ithaca) and 23.5 (its 2025 Aeneas-framework successor) to 15.5, and raises top-1 accuracy from 63.0 and 64.0 to 74.5.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07249
PDF
-
Structural Guidance for Unified Joint Demosaicing and Denoising
2026-08-07 10:14:20Qixin Zheng, Ping Chen, Qiangqiang Shen, Haijin ZengAbstract
Joint demosaicing and denoising is a fundamental step in camera image signal processing, yet remains challenging because different Bayer-like color filter arrays (CFAs) and sensor noise jointly corrupt both color sampling and image content. Existing unified restoration networks explicitly model CFA geometry but are still driven primarily by pixel-level supervision, making them prone to structural degradation around edges, repetitive textures, and moiré patterns where local evidence is unreliable. We attribute this limitation partly to the absence of explicit structural guidance beyond pixel-level reconstruction supervision. Motivated by this observation, we propose a structural-guided unified restoration framework that injects pretrained structural knowledge into CFA-aware image restoration. Our model receives a unified five-channel observation consisting of the raw mosaic, CFA masks, and a noise-level map. A SwinIR restoration branch reconstructs pixel details under CFA-conditioned modulation, while a parallel structural reasoning branch extracts complementary structural cues from a sparse pseudo-RGB observation. To bridge the substantial domain gap between sparse noisy sensor data and the natural-image pretraining domain of the structural encoder, we introduce a lightweight trainable adapter before residually fusing structural and restoration features. A shared decoder jointly predicts the restored RGB image and an auxiliary clean mosaic, providing supervision in both image and sensor domains. Extensive experiments across multiple CFA patterns and noise levels demonstrate consistent improvements over state-of-the-art unified and CFA-specific methods, indicating that adapted structural priors can enhance robust camera image restoration. The source codes and dataset are provided in the supplementary material.
Abstract (translated)
URL
https://arxiv.org/abs/2608.09995
PDF
-
Local Epistemic Uncertainty Guided Active Sampling for Plug-and-play Diffusive Image Restoration
2026-08-07 09:00:20Jiaqi Zhang, Zheng Pang, Rongrong Gao, Qiyuan Zhang, Yang YangAbstract
Diffusion models have demonstrated remarkable effectiveness in image restoration tasks. However, when guiding image reconstruction, existing Diffusion Model-based Image Restoration (DMIR) methods typically rely on fixed data constraints and uniform step sizes, thereby overlooking the dynamic nature of the generative process. Such rigid designs render the models vulnerable to spatially non-uniform degradations, thus resulting in structural distortions and loss of fine details. Meanwhile, uniform step sizes introduce computational redundancy, whereas naïve step reduction strategies tend to accumulate approximation errors. To address these limitations, we propose a Local Epistemic Uncertainty Guided Active Sampling framework (LEADer). In the spatial domain, LEADer leverages pixel-wise uncertainty to dynamically modulate the prior strength within the null space, which effectively balances detail preservation and artifact suppression. In the temporal domain, it quantifies sampling stability via the uncertainty trace to enable adaptive trajectory pruning, thereby accelerating convergence. Theoretical proofs demonstrate that our framework achieves strict data consistency, while the trajectory pruning strategy admits a deterministic error bound, thereby guaranteeing stable convergence under skip sampling. Notably, our plug-and-play method can be seamlessly integrated into various DMIR baselines. Extensive experiments show that LEADer improves the performance of multiple state-of-the-art DMIR methods, while significantly reducing sampling time with negligible memory overhead. Code is available at this https URL.
Abstract (translated)
URL
https://arxiv.org/abs/2608.06981
PDF
-
Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal
2026-08-07 08:13:40Wanshu Fan, Yunzhe Zhang, Yue Shen, Liyan Wang, Jing Qin, Kin-Man Lam, Cong Wang, Jinshan PanAbstract
Adverse weather causes diverse and complex image degradations, severely compromising the reliability of computer vision systems. Existing all-in-one restoration models attempt to address multiple degradation types within a unified framework, but often lack explicit spatial and semantic modeling of degradation characteristics, limiting their adaptability to diverse weather conditions. To address this limitation, we propose a Degradation-Aware Cross-Modal Prompt Compensation Network (DCMPC-Net) that leverages cross-modal degradation cues from a pretrained vision-language model to condition restoration features within a unified backbone. Specifically, our DCMPC-Net mainly consists of the Cross-Modal Prompt Generator (CMPG), Prompt-Guided Attention Alignment Module (PGAAM), and Dual Feature Compensation Module (DFCM). The CMPG integrates textual embeddings with visual features to produce degradation-aware prompts that encode degradation-related semantic and contextual cues. These prompts are injected into the decoder via a PGAAM, which adaptively aligns semantic information with degraded regions to facilitate context-aware restoration. To further enhance structural fidelity, DFCM is introduced that disentangles degradation artifacts from scene structures, thereby improving the reconstruction of fine textures and detailed content. By integrating cross-modal semantic guidance with spatial alignment and structural enhancement, DCMPC-Net achieves robust and perceptually consistent restoration across diverse weather conditions. Extensive experiments show that DCMPC-Net outperforms state-of-the-art methods in both task-specific and unified settings, achieving superior accuracy and visual fidelity.
Abstract (translated)
URL
https://arxiv.org/abs/2608.06939
PDF
-
HazeSpikeMamba: Coupling Spiking-Inspired and State-Space Features for Self-Supervised Real-World Dehazing
2026-08-07 07:16:11Haoran Liu, Huibin Li, Mingzhe Liu, Peng Li, Guibin ZanAbstract
Dehazing networks are commonly trained on synthetic hazy-clear pairs, but their performance often drops on real photographs. Synthetic haze generated using the atmospheric scattering model does not fully capture the variability of real haze, and paired real hazy-clear images are scarce. In this work, we propose HazeSpikeMamba, a compact dehazing framework that combines a spiking-inspired local path and an attentive state-space global path in a multi-scale U-Net. The local path uses TPCNNSpike, a new spike-emission scheme inspired by the neighborhood coupling of Pulse-Coupled Neural Network (PCNN). Unlike grouped directional scanning, TPCNNSpike updates all neurons in parallel using the previous firing states of their Gaussian-weighted neighborhoods. The global path adapts the Attentive State-Space Module of MambaIRv2, retaining semantic prompting and sequence reordering while removing the window self-attention branch. Its state-space processing models long-range dependencies with complexity linear in sequence length. For target-domain adaptation, a frozen degradation network, pretrained on paired NH-HAZE data, re-synthesizes haze from the dehazed prediction. The reconstruction error updates only the final restoration layers of HazeSpikeMamba without haze-free labels during adaptation. A shared checkpoint is adapted once on each complete unlabeled target set, making the evaluation dataset-level and transductive rather than zero-shot or per-image optimization. The forward network contains 2.02M active parameters and requires 13.27G nominal MACs (measured with thop at 256x256 input). This adaptation consistently improves BRISQUE and NIMA on RTTS, URHI, and HSTS. On RTTS, BRISQUE decreases from 30.13 to 27.72 and NIMA increases from 4.13 to 4.87. Under this transductive protocol, the adapted model also achieves the best BRISQUE and NIMA on URHI and HSTS among the compared methods.
Abstract (translated)
URL
https://arxiv.org/abs/2608.06886
PDF
-
TeXFix-Bench: An Empirically Grounded Multi-Format Benchmark for LLM-Based Document Source Repair
2026-08-07 05:56:35Prajwal S. VenkateshmurthyAbstract
Scientific and technical writing depends on markup sources that must compile: LaTeX, Typst, and Markdown pipelines fail on missing delimiters, mismatched environments, broken imports, or package conflicts. Existing document-repair evaluations inject faults with ad-hoc edits that lack an empirical fault model. We present TeXFix-Bench, a multi-format benchmark for LLM-based full-source document repair grounded in a mined fault taxonomy. A Grounded-Theory study of localized hard-crash LaTeX faults from TeX Stack Exchange, GitHub commits, and package documentation (168 verified faults, dual open coding at $\kappa$=0.34) yields an 18-category taxonomy instantiated as DocMut: 48 AST-aware operators across three formats. A three-model cross-benchmark shows DocMut faults are 5.6-9.2 pp harder to repair than pattern-based mutations on the same seeds, and a real-error case study (88 mined human crashes, 67.0% repair success) brackets both synthetic sets from below. We construct 10,437 instances from 743 openly licensed seeds and evaluate seven LLMs under a fixed zero-shot protocol with provider-pinned routing, collecting 48,651 attempts at about USD 200 total inference cost. A complete 6,613-instance x 7-model balanced matrix confirms all rankings. A pinned engine gate yields a 27.5-point intention-to-treat compile spread (56.7-84.2%). Typst is markedly harder than LaTeX and Markdown. A restoration oracle over 28,129 compiling repairs shows that 13.6-18.5% of compiling repairs materially alter document text, and restoration rank diverges from compile rank: the model with the lowest compile rate restores content best among its successes. Compile success alone overstates repair quality. We release the taxonomy, DocMut, and all campaign artifacts.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07617
PDF
-
Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration
2026-08-07 05:44:31Zihao He, Yunfeng Wu, Xinchao Wang, Songhua LiuAbstract
All-in-one image restoration seeks a single model that can recover images degraded by diverse and spatially non-uniform corruptions. However, many unified Transformers rely on fixed patch partitioning: task/degradation condition is injected only into the backbone blocks after tokenization, leaving the embedding and reconstruction stages insensitive to local degradation variations. In contrast to previous approaches, we present Flexible Image Transformer (FIT) that explicitly models degradation awareness across the entire pipeline, from patch sampling to pixel reconstruction. Specifically, FIT employs a lightweight Degradation Encoder to predict a global degradation vector $\mathbf{g}$ and a spatial degradation map $\mathbf{M}$ from local degradation severity, which jointly condition the patch embedding and unembedding through adaptive deformation. Moreover, to improve robustness across degradation types, we introduce a task-token dropout strategy that regularizes task conditioning during training. On five standard benchmarks (BSD68, Rain100L, SOTS, GoPro, and LOLv1), FIT achieves state-of-the-art performance with 30.72 dB average PSNR on the five-degradation setting and 32.83 dB on the three-degradation setting, outperforming recent unified restoration methods by +0.5$\sim$1.1 dB. Moreover, the learned offsets provide a direct handle for visualizing degradation-aware spatial adaptation.
Abstract (translated)
URL
https://arxiv.org/abs/2608.06832
PDF
-
WaveFreqAnchor: Wave-Structural Anchoring and Frequency Correction Diffusion for Training-Free Face Restoration
2026-08-07 02:23:25Zelin Du, Wenjie Li, Zhengxue Wang, Juncheng Li, Cailing Wang, Guangwei GaoAbstract
Diffusion-based face restoration that adjusts the sampling trajectory of pre-trained diffusion models has achieved remarkable progress. However, existing approaches provide insufficient constraints during reverse diffusion, causing identity-related structural drift and degraded fidelity under severe degradations. To address this, we propose WaveFreqAnchor, a training-free framework based on Wave-Structural Anchoring and Frequency Correction Diffusion. Specifically, Anchor-Space Wave-Structural Guidance (ASWG) constrains facial structures through anisotropic wave-response consistency, while Multi-scale Wavelet-Fourier Injection (MWFI) aligns the predicted low-frequency subband with the observation by replacing its phase, correcting inconsistencies accumulated during reverse diffusion. For real-world scenes, we further introduce Subband High-Frequency Enhancement (SHE), which performs bounded, spatially masked refinement on the predicted high-frequency subbands to recover fine facial details under unknown compound degradations. Together, these designs effectively preserve facial identity while restoring sharp and realistic facial details. Extensive experiments show that our method consistently outperforms existing methods, achieving high-quality and high-fidelity face restoration.
Abstract (translated)
URL
https://arxiv.org/abs/2608.06717
PDF
-
DTOC: Dynamic Tool Output Compression for Adaptive Context Management in AI Agents
2026-08-06 12:06:36Abhay Chaturvedi, Shreya Bhattacharya, Rashmika Gopalkrishnan, Peter van der PuttenAbstract
As agent capabilities have grown, practical limitations increasingly stem from constrained context windows rather than model capacity. Common strategies, such as truncation, heuristic aging, and lossy summarization, may discard useful information or introduce hallucination risk. To address these challenges, we propose Dynamic Tool Output Compression (DTOC), a framework for scalable context management in LLM-based agents that models context updates as explicit and reversible operations within the agent reasoning loop. DTOC retains full tool outputs in external memory while inserting compact placeholders into the active context, enabling selective reconstruction when needed. We formalize the DTOC mechanism, integrate it into a ReAct-style agent architecture, and provide a production-oriented implementation supporting on-demand restoration of compressed outputs. Experiments on DeepSWE reveal model-dependent effects: for responsive models (Sonnet 4.6, GPT-5.4), DTOC reduces input tokens (10.3 and 12.7%) and agent steps (2.4 and 32.3%), while increasing solve rates (2.5 and 1.5 times higher) and lowering cost per solved task (3 and 3.5 times lower cost per solved task). For the other models results are more mixed, with GPT-5.5 doubling solve rate and halving cost, but no impact on solve rate and negative impact on cost for the other models. Ablation results show reversibility is critical: disable-only compression variants degraded performance, while full DTOC recovered baseline accuracy at substantially lower context cost. These findings indicate that explicit, reversible context management can improve the efficiency of long-horizon agent reasoning without degrading task performance.
Abstract (translated)
URL
https://arxiv.org/abs/2609.26121
PDF
-
NewtonGS: Physics-Structured Object-Level Neural Newtonian Dynamics for Gaussian Scene Animation
2026-08-06 09:41:43Lianlei Shan, Feiyang Ye, Yan Chen, Yong WuAbstract
Animating objects in a static 3D Gaussian scene requires an explicit object-level dynamic state and a controllable model of object motion. Existing dynamic Gaussian methods primarily reconstruct time-varying scenes or simulate deformation, rather than provide compact object states for direct control. To address this gap, we present NewtonGS, a physics-structured framework for object-level state rollout and Gaussian scene animation. NewtonGS represents each object with a 22-dimensional state covering pose, linear and angular velocity, anisotropic scale and its rate, mass, and contact properties. Its Gaussian Neural Newtonian Dynamics (Gaussian-NND) model combines analytic translation, quaternion kinematics, gravity, damping, and scale-restoration dynamics with learned continuous and contact residuals. A discrete event map handles floor contact. Predicted poses and scales define a shared affine transformation that updates the means and covariances of all Gaussians associated with each object. We construct two procedurally generated datasets: State-32 for state-rollout evaluation and Gaussian-32 for state-to-Gaussian transformation. On both the in-distribution and velocity-range-shift splits of State-32, NewtonGS achieves lower trajectory RMSE, final displacement error, and velocity RMSE than five analytic baselines. Experiments on Gaussian-32 further demonstrate effective conversion from predicted states to animated Gaussian objects.
Abstract (translated)
URL
https://arxiv.org/abs/2608.07598
PDF
-
Flow-Map Distillation on Relation Manifolds for Image Restoration
2026-08-06 08:59:28Zihao He, Songhua LiuAbstract
Knowledge distillation for image restoration typically aligns intermediate features or relation matrices between teacher and student networks as static targets, ignoring the dynamic structure of the knowledge transfer process. In this paper, we propose Flow-Map Distillation on Relation Manifolds (FoRM), which reformulates relation-based knowledge transfer as a continuous flow mapping problem on the relation manifold. Rather than regressing a constant velocity field between student and teacher relation states, FoRM learns a flow map operator $\mathcal{F}_\theta(\mathbf{z}, t, s)$ that directly predicts the relation state at any target time $s$ given the current state at time $t$, enabling richer trajectory-level supervision. To ensure global self-consistency of the learned flow map, we introduce a safe semigroup consistency constraint that enforces compositional agreement using ground-truth bridge states, eliminating phantom-state error accumulation. An endpoint anchoring loss further prevents the operator from drifting away from the teacher target. Extensive experiments on five image restoration tasks, including super-resolution, deraining, denoising, deblurring, and low-light enhancement, demonstrate consistent gains over state-of-the-art distillation baselines across multiple backbone architectures, reducing training variance by approximately 50\% compared to naive flow matching distillation while achieving superior restoration quality.
Abstract (translated)
URL
https://arxiv.org/abs/2608.05769
PDF
