Embedding
-
SMI: Ontology-Grounded Semantic IDs for Hierarchy-Aware Medical Concept Representation
2026-10-13 16:30:01Ziyang Song, Lia Shen, Yixuan Li, Qincheng Lu, Weihao Li, Haoran Xu, He Zhu, Xinyu Wang, Yue LiAbstract
Biomedical language models (LMs) provide strong representations for clinical tasks, but they often fail to capture the hierarchical structure among medical concepts. This limitation is especially pronounced for rare and fine-grained clinical codes in long-tailed vocabularies, where hierarchy-agnostic LM representations become less informative as concept frequency decreases. We introduce Semantic Medical Identifier (SMI), a training-free framework for constructing ontology-grounded Semantic IDs and hierarchy-aware medical concept representations. For each diagnosis, procedure, or medication concept, SMI defines its Semantic ID as the coarse-to-fine path through an expert-defined medical ontology. SMI then aggregates frozen biomedical LM embeddings across the corresponding ontology levels to form hierarchy-aware medical concept representations that preserve coarse-to-fine clinical semantics. The base SMI uses uniform aggregation across ontology levels, while a level-weighted variant selects global level weights on validation data. Our theoretical analysis shows that SMI yields more informative medical concept representations than hierarchy-agnostic LM embeddings, particularly for rare and fine-grained clinical concepts. Experiments show that SMI captures meaningful hierarchical structure and improves clinical relation prediction. On MIMIC-IV, SMI improves mortality prediction, readmission prediction, drug recommendation, and rare drug prediction. On eICU, SMI yields more robust patient representations under cross-hospital distribution shift.
Abstract (translated)
URL
https://arxiv.org/abs/2610.18672
PDF
-
Graph-free Retrieval with Graph-Modified Embeddings
2026-10-13 13:51:46Federica Santi (Cognee Inc), Vasilije Markovi\'{c} (Cognee Inc), Daniel Scott (Cognee Inc), Lazar Obradovi\'{c} (Cognee Inc)Abstract
Dense vector retrieval can miss evidence connected through intermediate entities, while direct graph retrieval requires access to the graph at query time. We compare five offline graph-based embedding transformations and strategies for using the resulting graph-modified embeddings alongside the originals, without query-time graph access. On a synthetic professional-network benchmark with complete supporting-evidence sets, direct replacement of the original embeddings reduces evidence recovery. Multi-stage strategies improve recovery by using graph-modified embeddings to guide the search while retaining the originals for semantic relevance. On questions requiring linked facts or comparisons among candidates, the strongest configuration improves recovery by up to 25.9% relative to plain vector retrieval. It also matches baseline recovery with 40% fewer returned nodes. Controls confirm that graph information contributes beyond multi-stage retrieval alone. Across the tested methods, retrieval performance depends more on how graph-modified embeddings are used than on how they are produced.
Abstract (translated)
URL
https://arxiv.org/abs/2610.18410
PDF
-
Choosing LLMs for Strategic Agent Interactions
2026-10-13 13:21:27Arastun Mammadli, Herbert Woisetschl\"ager, Jawad Fayaz, Shiqiang WangAbstract
Game-theoretic benchmarks claim that large language model (LLM) strategic decision-making varies across game types, interaction structures, and capability dimensions. However, they only evaluate strategic performance across individual fixed models. In parallel, contemporary LLM routers pick models under quality-cost trade-offs, but with no notion of a strategic structure. They do not account for an opponent that might be unknown, and that can adapt over time and shift the feedback signal. To address this, we extend the game-theoretic evaluations and define strategic selection as the meta game layered on top of the base game. In each round, a strategic selector fields an LLM from a portfolio of models to navigate a game payoff matrix that is latent and learned online. We find that the evaluation signal is game-specific, and that extracting a reliable signal requires a much higher number of seed runs $N$ compared to traditional benchmarking setups. Evaluating across 5 games and 10 LLMs, we show a model's strategic competence varies significantly depending on the game setup and the opponent faced. Building on this, we implement a suite of selector methods and find that approaches that rely on query embeddings or LLM self-reflection do not consistently capture the underlying strategic signal. A lightweight multi-armed bandit algorithm performs more reliably across games, though it still falls short of the best fixed model. Finally, we show that simple non-stationary and adversarial bandit variants fail to outperform a stationary bandit baseline, pointing to a gap in how current methods handle a shifting opponent in strategic settings.
Abstract (translated)
URL
https://arxiv.org/abs/2610.18354
PDF
-
ShoeFit: A New Dataset and Dual-image-stream DiT Framework for Virtual Footwear Try-On
2026-10-13 13:03:29Yuhan Li, Zhiyu Jin, Yifan Tong, Wenxiang Shang, Benlei Cui, Xuanhong Chen, Hangcheng Zhu, Bingbing NiAbstract
Virtual footwear try-on (VFTON), a critical yet underexplored area in virtual try-on (VTON), aims to synthesize faithful try-on results given diverse footwear and model images while maintaining 3D consistency and texture authenticity. Unlike conventional garment-focused VTON methods, VFTON presents unique challenges due to (1) Data Scarcity, which arises from the difficulty of perfectly matching product shoes with models wearing the identical ones, (2) Viewpoint Misalignment, where the target foot pose and source shoe views are always misaligned, leading to incomplete texture information and detail distortion, and (3) Background-induced Color Distortion, where complex material of footwear interacts with environmental lighting, causing unintended color contamination. To address these challenges, we introduce MVShoes, a multi-view shoe try-on dataset consisting of 7305 well-annotated image triplets, covering diverse footwear categories and challenging try-on scenarios. Furthermore, we propose a dual-stream DiT architecture, ShoeFit, designed to mitigate viewpoint misalignment through Multi-View Conditioning with 3D Rotary Position Embedding, and alleviate background-induced distortion using the LayeredRefAttention which leverages background features to modulate footwear latents. The proposed framework effectively decouples shoe appearance from environmental interferences while preserving high-quality texture detail through decoupled denoising and conditioning branches. Extensive quantitative and qualitative experiments demonstrate that our method substantially improves rendering fidelity and robustness under challenging real-world product shoes, establishing a new benchmark in high-fidelity footwear try-on synthesis.
Abstract (translated)
URL
https://arxiv.org/abs/2610.18325
PDF
-
FALLNEST: Dual-Domain Dataset Condensation of Imbalanced IMU Data for Wearable Fall Detection
2026-10-13 12:25:20Rashi Niyas P, Ritika Jain, Hitika TiwariAbstract
Wearable fall detection is hampered by rare fall events, scarce labeled IMU data, and the cost of training deep models on large, imbalanced corpora. Existing methods either rely on hand-crafted features with shallow classifiers, which limits learning from raw signals, or train deep networks on full datasets that require re-sampling to counter class skew and demand substantial data and compute. To address this, we propose a novel Fall detection framework via Normalized-gradient and Embedding matched Synthetic Time-frequency condensation, dubbed FALLNEST, a data-centric approach that distills large, imbalanced IMU datasets into tiny, class-balanced synthetic training sets. Building on dual-domain time-series condensation (CondTSC), FALLNEST introduces a fall-specific 7-channel input with accelerometer vector magnitude, class-balanced K-Means initialization, and normalized gradient matching with gradient clipping, combined with MMD-based embedding alignment and multi-view Fourier-domain augmentation, so that class balance arises from the condensed set itself rather than from re-sampling. We evaluate on SisFall, KFall, and MobiFall using strict subject-wise splits and naturally imbalanced test sets, reporting accuracy, Macro-F1, and PR-AUC over five seeds. A 102K-parameter TinyCNN trained on only 32 to 146 synthetic windows (about only 1% of the training windows) reaches 93.5-95.8% mean accuracy and 0.89-0.98 PR-AUC, with standard deviations of 1.0-1.7% in accuracy, demonstrating the efficacy of our approach.
Abstract (translated)
URL
https://arxiv.org/abs/2610.18260
PDF
-
Stabilizing the Modality Gap: Null-Space Projected Continual Learning for Vision-Language Models
2026-10-13 11:41:24Tiantian Peng, Yuyang Liu, Shuo Yang, Qiuhe Hong, YongHong TianAbstract
The modality gap is intrinsic to multimodal alignment, where embeddings from different modalities remain partially separated in a shared representation space. In continual learning, CLIP-based vision-language models (VLMs) not only suffer catastrophic forgetting but also experience modality gap drift that disrupts the shared embedding geometry. Such drift degrades cross-modal alignment and generalization. We propose FPCD, an efficient framework combining Fast Null-space Projection (FNP) and Contrastive Distillation (CD). FNP accumulates compact Gram matrices of previous tasks' activations and projects gradients onto layer-adaptive approximate null spaces to reduce interference with learned representations. By replacing singular value decomposition with symmetric eigendecomposition, FNP accelerates projection-basis construction by a factor of 5-9. CD uses a small reference set to align the bidirectional image-to-text and text-to-image similarity distributions of the fine-tuned model with CLIP, thereby preserving pretrained cross-modal relations. On the 11-task MTIL benchmark, FPCD achieves 87.6% Last, 76.4% Average, and 65.2% Transfer. It also maintains more stable modality gaps and stronger zero-shot retrieval performance on COCO Captions and Flickr30k than competing methods. These results highlight the importance of controlling modality gap drift during continual adaptation of VLMs.
Abstract (translated)
URL
https://arxiv.org/abs/2507.19839
PDF
-
Beyond Black Boxes: Visual Grounding for Text Embeddings
Abstract
In this paper, our goal is to represent text embeddings as sketches for three main reasons. First, to visualize text representation and to explore their potential their potential for AI explainability. Second, to investigate the potential benefits of the sketch-based representation for information retrieval. Third, to explore their potential for reasoning, drawing inspiration from the concept of mental imagery. We propose a sketch-based visual grounding framework that transforms text representations into visual topographic sketches. Our approach aims to emphasis the semantics content encoded in the text embeddings through the corresponding sketch. Evaluations are conducted on the MS-COCO dataset to examine the utility of these sketches for text retrieval. The comparative study of SBERT and Sketch-based method reveals that SBERT retains an advantage in strict top-1 retrieval accuracy while achieving comparable performance to the sketch-based method in top-5 retrieval. A cascaded approach combining SBERT and the sketch-based method outperforms SBERT alone, suggesting that the Sketch-based representation could offer significant potential for information retrieval if trained directly on text.
Abstract (translated)
URL
https://arxiv.org/abs/2610.18047
PDF
-
Better Capturing Interactions Between Products in~Retail: Revisited Negative Sampling for~Basket Choice Modeling
2026-10-13 08:41:22Jules D\'esir, Vincent Auriau, Martin Mo\v{z}ina, Emmanuel MalherbeAbstract
Brick-and-mortar retailers face many different challenges that involve understanding thoroughly its products catalog and customer preferences. In particular, assortment optimization - proposing the ideal mix of products - and promotion planning hold a pivotal role in their strategy. By leveraging sales data, retailers can make informed decisions on which products to sell and how to manage inventory, based on customer preferences as well as regional and seasonal trends. It is especially crucial to capture interactions between products, in order to minimize the number of items that cannibalize each other's sales and to ensure that complementary products, which are often purchased together, are conjointly available and sold across all stores. In this paper, we propose a model of shopping basket that learns embeddings to represent interactions between products, prices and stores. Our model is built to uncover sales patterns from very large transaction datasets. In particular, the optimization loss is computed with random negative samples in order to overcome the computational bottlenecks that arise with large number of items. Our experiments on synthetic data show the efficiency of drawing such negative samples based on the actual assortment of available products, with better results than approaches from the literature. We also validate our approach by training and evaluating our model on a dataset composed of millions of transactions from Fortenova Group, one of the leading retail company in Southeast Europe. Our model showcases promising applications in the sector of retail, with enriched interfaces to efficiently support category managers.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17963
PDF
-
A Tutorial on Quantum Amplitude Embedding with Data Re-Uploading
Abstract
Amplitude embedding lets a variational quantum model read many classical features with few qubits, but the model output is then only a quadratic form in the normalized input. Data re-uploading, which encodes the input repeatedly between trainable blocks, can be used to enrich such a model. Because every additional upload of an amplitude embedding repeats a costly state preparation, the gain that an upload buys has to be known before the gates are spent. This tutorial thus addresses the problem of explaining what data re-uploading adds to an amplitude-embedded model and how large the gain is. We express sequential re-uploading and parallel re-uploading with two copies in one notation and show that each upload raises the reachable polynomial order by two. Under one trainable circuit family and one budget of trainable parameters, simulation results show that re-uploading raises the test coefficient of determination from 0.689 to between 0.883 and 0.975 on targets of order four and from 0.337 to between 0.642 and 0.783 on targets of order six, on all ten seeds. On a quadratic target and on handwritten digits, one upload remains the best choice, and every additional upload lowers the gate error that the classifier tolerates. These results help designers match the number of uploads to the polynomial order of the task at hand.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17836
PDF
-
FLOMSS: Generative Source Separation by Flow Matching
2026-10-13 06:43:47Shuyang Cui, Zhi Zhong, Keisuke Toyama, Koichi Saito, Qiyu Wu, Yukara Ikemiya, Junghyun Koo, Kin Wai Cheuk, Woosung Choi, Giorgio Fabbro, Wei-Hsiang Liao, Shusuke Takahashi, Yuki MitsufujiAbstract
Music source separation (MSS) has been conventionally addressed by discriminative methods. Recently, flow matching has emerged as a strong generative paradigm for modeling complex data distributions, with successful applications across diverse domains. These advances motivate us to systematically investigate flow matching for MSS in a four-source (Vocals, Bass, Drums, Other) setting in full-band stereo. Specifically, we investigate prediction targets, flow paths, single-source versus multi-source modeling, and mixture consistency constraints. Beyond the signal-level metrics commonly used for MSS, such as signal-to-distortion ratio, we further evaluate MSS methods using embedding-based metrics, as well as a listening test, providing a comprehensive benchmark. On MUSDB18-HQ, our proposed model exhibits competitive performance in both objective and subjective tests, highlighting the potential of flow matching for generative MSS. Demo: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2610.17821
PDF
-
SSEmb: Training a Generalizable Speech Retrieval Model Using Only Automatic Speech Recognition Data
2026-10-13 06:39:50Sangkwon Park, Dongwook Lee, Youngho Cho, Donghun Kang, Heeseung Kim, Sungroh YoonAbstract
End-to-end speech-to-text retrieval is becoming an important capability for voice agents, allowing spoken queries to retrieve relevant documents without intermediate transcription. Training these retrievers typically depends on large collections of spoken query-document pairs that are costly to acquire and difficult to extend across domains and languages. We introduce \ours, which replaces such task-specific supervision with speech-transcript pairs from standard ASR corpora. A lightweight adapter projects the output of a frozen speech encoder into the retrieval space of a frozen pretrained text embedding model. \ours is competitive with cascaded ASR-then-retrieve systems while reducing inference latency, and outperforms existing publicly available speech embedding models. It also transfers to text-to-speech and speech-to-speech retrieval without additional training. Controlled analyses validate the speech-transcript alignment paradigm, demonstrating robustness to reduced training data and consistent gains from larger corpora. Further ablations of loss functions and trainable modules establish practical guidelines for the framework. Model weights and codes are to be released.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17813
PDF
-
A Node Embedding Framework for Model-Centric Conformance Checking in Process Mining
2026-10-13 04:34:39Mostafa Abbasi, Homayoun NajjaranAbstract
Evaluating the quality of process models discovered from event logs is a fundamental challenge in process mining. Traditional evaluation relies on conformance metrics such as fitness and precision, which are highly sensitive to noise and deviations in event logs and therefore often fail to give a consistent assessment of discovery algorithms. We introduce a model-centric evaluation framework that combines node embeddings with Procrustes alignment to enable cross-graph comparability, since embeddings derived from different graphs reside in distinct vector spaces. The proposed PetriNetGraph method transforms Petri net process models into graph representations compatible with embedding algorithms. Comparing the node embeddings of ground-truth and discovered models through a parameterized cosine distance yields a unified and bounded metric for evaluating process discovery across datasets and methods. We evaluate the framework under static and context-aware scenarios to examine how prior knowledge of activity existence influences the assessment of structural similarity. The framework also integrates a compliance checking mechanism that localizes deviations at the activity level and an entropy-regularized node distance aggregation function that quantifies global deviations across the model. Experiments on nine publicly available event logs show that the approach provides a reliable embedding-based conformance indicator that reflects the behavior of fitness and precision. Experiments on synthetic logs with controlled noise levels confirm that it is substantially more robust to log deviations than traditional conformance metrics, PetriNet2Vec, and its dynamic-window and Procrustes-aligned variants, while remaining sensitive to process model degradation. All implementation details and datasets are publicly available.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17702
PDF
-
The Tokens Strike Back: Safety-Aware Denoising with Directional Embedding-based Revision
2026-10-13 03:04:22Amman Yusuf, Jiwoo Shin, Ruizhi Deng, Heitor Rodrigues Guimar\~{a}es, Nikita Dvornik, Mijung ParkAbstract
The Safety-Aware Denoiser (SAD) is a recent, training-free, safety-guidance framework for text diffusion models. During inference, SAD steers text samples toward safe regions of the token probability space. In this paper, we propose $\mathit{SADDER}$ (Safety-Aware Denoising with Directional Embedding-based Revision), which improves the safety and quality trade-offs of SAD by exploiting a safety-guidance signal in the embedding (or activation) space for each token position, then updating the token positions using the SAD update rule multiplied by the guidance signal. This multiplication acts as a gating mechanism: tokens with a high unsafe signal are more likely to be modified during inference following the SAD update rule. However, such an update does not automatically preserve the conditional mean, i.e., the denoiser of a fixed target distribution. We therefore theoretically derive new position-wise target distributions for which the modified denoiser is exact. Furthermore, we derive conditions under which the position-wise target distributions match a global target distribution. We test our $\mathit{SADDER}$ method on popular Masked Diffusion Language Models (MDLMs) and Uniform-State Diffusion Models (USDMs) across unsafe continuation and jailbreak settings, and show that $\mathit{SADDER}$ achieves a better balance between safety, generation quality, and inference cost than existing methods.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17636
PDF
-
Chasing Label Shifters: A Change-Aware Framework for Dynamic Graph Node Classification
2026-10-13 02:00:08Junghoon Kim, Seungyoon Choi, Hyunsung Kim, Chanyoung ParkAbstract
Entities in real-world systems often evolve over time: users shift between information consumption patterns, researchers migrate between fields, and firms transition between financial risk states. When such systems are modeled as dynamic graphs, these transitions correspond to nodes whose class labels change between consecutive snapshots, which we call shifters. Correctly classifying shifters is often more consequential than classifying stable nodes with persistent labels, as detecting such transitions enables timely intervention in high-stakes settings. Yet, we identify a systematic failure mode across all major dynamic graph neural network architectures: shifter performance consistently lags behind stable nodes, with errors concentrated on the old label. Our structural analysis traces this failure to embedding inertia: neighborhood aggregation keeps shifter representations anchored to the old-class centroid, and this effect is strongest precisely where it is hardest to correct, namely for shifters whose neighborhoods remain aligned with the old class. Guided by this finding, we propose CHASE, a model-agnostic CHange-Aware framework for Shifting nodEs that wraps any dynamic GNN with targeted components for detecting label shifts and overriding stale neighborhood signals. CHASE consistently improves shifter performance across all tested models and datasets (up to 114.9% and 202.5% improvement on accuracy and F1 score, respectively) while preserving stable-node performance. We additionally contribute three dynamic graph benchmarks with naturally shifting labels, filling a gap in existing resources. The source code is available at this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2610.17582
PDF
-
Foundation Models for Event Prediction Across Heterogeneous Event Logs
2026-10-13 00:00:04Fabian Spaeh, Sean Jaffe, Shandian Zhe, Bin ShenAbstract
Event logs are ubiquitous across real-world domains, including business processes, manufacturing workflows, healthcare, and many other operational systems. Yet event logs can differ substantially in their event semantics, label spaces, interaction patterns, and temporal scales, making it difficult to build a single foundation model that generalizes directly to previously unseen systems. We address this challenge with the Foundation Event Predictor (FEP), a simple and scalable in-context architecture that performs next-event prediction through learned, event-level nonparametric inference. Causal self-attention encodes context and target histories into key and query representations, respectively, while the observed next context events are encoded as values. Cross-attention then performs learned, relevance-weighted aggregation of next-event outcomes over all context transitions, allowing the entire nonparametric prediction procedure to be expressed as standard attention operations and efficiently parallelized with modern kernels such as FlashAttention. To promote cross-system generalization, we introduce permutation-robust embedding learning and random fixed embeddings to decouple event IDs from persistent semantics, together with empirical quantile transforms and robust normalization to handle diverse temporal scales. We pretrain FEP on 696 real-world event datasets spanning heterogeneous event systems and evaluate it on 50 held-out datasets unseen during pretraining. Across these datasets, FEP generalizes effectively to new event systems without parameter adaptation, frequently approaching the performance of strong models trained separately on each target dataset while substantially outperforming existing foundation-model baselines, particularly on challenging prediction tasks.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17503
PDF
-
Know-Thy-Not: Training-Free Targeted Negation in Dual-Encoder Vision-Language Models
2026-10-12 21:58:20DoYoung Kim, Changkyu Choi, Ad\'in Ram\'irez Rivera, YunSeok Choi, Jee-Hyong LeeAbstract
Dual-encoder vision-language models systematically fail at negation, scoring ''a cat without a dog'' nearly identically to ''a cat with a dog.'' We show that this failure is inherent to the single-embedding framework. Under contrastive training, single-embedding methods can at best ignore the negated concept (dilution) but never penalize its presence (rejection). We propose Know-Thy-Not (KTN), a training-free method that introduces an image-side rejection dimension to penalize the negated concept directly in the visual representation. We also introduce COCO-Confusion, the first benchmark requiring genuine rejection to succeed. Across 9 backbones, KTN improves COCO-Confusion mAP by up to +25.6 over vanilla VLMs and +9.6 over the strongest existing method.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17440
PDF
-
Large Language Models Extend Known Research But Do Not Create New Research Directions
2026-10-12 21:57:24Zhihua Liu, Kurt Butler, Damian Machlanski, Danilo Bueno, Sotirios A. TsaftarisAbstract
The success of autonomous research systems relies on the quality of the scientific hypotheses they generate. Recent studies report that human reviewers often rate LLM-generated ideas as more novel than those from human experts, raising hope that AI can accelerate scientific discovery. Yet it is unclear whether these models can do more than extend existing knowledge. Can LLMs drive genuine innovation by revealing new research directions, or are they limited to repeating and recombining what science already knows? This unresolved question lies at the heart of discussions about AI and the future of science. To address this, we distinguish two types of novelty: distance, which measures how far a hypothesis lies from existing work, and direction, which captures whether the hypothesis pursues new lines of inquiry beyond those already present. These two measures diverge sharply for LLM-generated work. We represent scientific papers in a shared conceptual space using a pretrained scientific-paper embedding model. Within this space, we embed 26.9 million human-authored papers across 1,325 research topics in six natural-science fields and 524,308 LLM-generated hypotheses from eight models. Distance is measured using existing metrics. To quantify direction, we introduce the orthogonal novelty score (ONS). LLMs consistently generate ideas more distant from existing work than human-authored papers, up to 16\% further when the LLM is given the papers farthest from the centre of the topic. However, LLM hypotheses have a 15--23\% lower median ONS than published papers from the same topic. Thus, LLMs move further from what is already known, but remain within established directions. This pattern persists across changes in various configurations. As a result, LLMs excel at extending the frontier but the creation of new directions is still led by human scientists.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17439
PDF
-
Learning to Detect Shifts in Unstructured Data Streams via Monitoring-by-Betting
2026-10-12 21:05:26Siyu Zhou, Jialin Ouyang, Fan Xia, Adarsh Subbaswamy, Drew Prinster, Suchi Saria, Augusto Garcia-Agundez, Sarah Pollet, Jinoos Yazdany, Patrick Vossler, Jean FengAbstract
Continuous monitoring of generative AI systems requires detecting distribution shifts in the unstructured data they ingest and generate. Existing monitoring methods typically control false alarms by requiring the discrepancy measure used to detect shifts to be *pre-specified*. However, for unstructured data, pre-specification creates a sensitivity--breadth trade-off: a narrowly defined measure is sensitive to the anticipated shift but may miss unexpected changes, whereas a broadly defined measure may detect a wider range of shifts but with reduced sensitivity. Although adaptively hypothesizing the shift direction can overcome this limitation, such data-dependent adaptation generally invalidates conventional false-alarm guarantees. We reconcile adaptivity with statistical validity through testing-by-betting, reframing sequential changepoint detection as a betting game in which the bettor continually updates its hypothesized shift direction, or witness function, as new observations arrive. We introduce *asymptotic e-detectors*, which approximate the pre-change distribution using a calibration set of size $m$, and prove that they asymptotically control the average run length to a false alarm as $m$ increases. Empirically, calibration sets as small as $m=30$ provide effective control. We then develop adaptive monitoring strategies ranging from black-box bettors that learn from embeddings to interpretable bettors based on LLM-proposed concepts. Across three real-world datasets, including data from a clinical AI scribe deployment, adaptive monitors detect shifts faster than fixed monitors at the same false-alarm level. Further, when the true shifts are sparse and interpretable, LLM-concept monitors provide the fastest and most interpretable detection, while also guiding root-cause diagnosis for complex shifts.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17399
PDF
-
Budgeted KV-Cache Repair for Evolving Contexts
2026-10-12 19:44:57Han Yu, Wenhui Zhu, Jincheng Cao, Rui Cai, Yuanda Xu, Zhengze Zhou, Xiwen Chen, Hao Wang, Xuanzhao Dong, Han Shi, Menglin Zhou, Shayan Mohajer Hamidi, Lin Su, Mina Razghandi, Sayan Ghosh, Qing Lan, Rohit Jain, Alborz GeramifardAbstract
Language-model agents revise their working context as information becomes obsolete, but deleting text does not erase its influence on cached computation. Later key-value (KV) states remain conditioned on the deleted content, while recomputing the entire retained suffix can be costly. We introduce budgeted KV-cache repair: recovering task performance by recomputing a user-chosen fraction of the retained suffix. Comparing repair allocations under identical relevance scores and token budgets, we find that refreshing coherent regions can substantially outperform refreshing isolated tokens. This finding motivates Semantic-Unit Refresh (SUR), which allocates the budget to query-relevant text regions using inexpensive embedding scores, without training or an auxiliary transformer forward. To execute these repairs efficiently, a layer-packed executor processes selected states together at each layer while preserving full causal cache access. Across five models, SUR achieves strong recovery in controlled deletion tasks while recomputing only 2-5% of the suffix. On four natural multi-hop QA datasets, it achieves near-full-recompute exact match at a 5% budget with lower post-edit latency. The executor accelerates fragmented refresh by up to 16.1x over interval-wise execution.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17338
PDF
-
Joint Embedding with Dynamic Integration network
2026-10-12 19:13:24Rishav Mukherjee, Jeffrey Ahearn ThompsonAbstract
Deep learning has become the dominant form of machine learning due in part, to its ability to automatically engineer complex meta features from data. However, even deep learning features can perform erratically because of unobserved latent factors. incorporating latent factor information explicitly, can improve performance by letting the model alter its behavior based on such inferred factors. Because neural networks often capture such complexity inherently, external clustering of the original feature space offers limited benefit. clustering observations by feature importance profiles, rather than raw features, offers a principled way to uncover latent structure, since observations with similar importance patterns may share predictive dynamics. In a prior work, Feature Oriented Representation with Clustering and Explanation (FORCE) we showed that this approach measurably improved neural network performance. In this article, we present JEDI-net(Joint Embedding with Dynamic Integration Network) a unified architecture that periodically clusters SHAP values during optimization to identify latent subgroups with similar feature importance patterns, embed their identities directly into the network, allowing predictions to adapt to each observations predictive structure.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17314
PDF
