Knowledge
-
Cross-Lingual Transfer of Evil Vectors Is Directionally Asymmetric
Abstract
Persona vectors are linear directions in the residual-stream activations of LLMs that encode behavioral traits and enable fine-grained control over model behavior. Among them, the evil persona vector is particularly relevant to understanding harmful behavior, jailbreak vulnerabilities, and the unlearning of harmful knowledge. Yet whether this vector reflects a genuinely cross-lingual concept, rather than an artifact of its extraction language, remains untested. We conduct a systematic evil-vector transfer study across two high-resource languages (English, Spanish) and two low-resource languages (Swahili, Telugu) on four open-source instruction-tuned LLMs, and find that transfer is directionally asymmetric. Evil vectors are harder to obtain reliably from low-resource languages, and the resulting vectors are largely ineffective at steering, even within their own language. In contrast, vectors extracted from high-resource languages transfer effectively to low-resource inputs while largely preserving general knowledge, as measured by MMLU. Geometric analysis further supports this pattern: evil vectors from the two high-resource languages are closely aligned, whereas low-resource vectors align only weakly with this shared direction. These results suggest that a shared cross-lingual evil direction exists and, once reliably extracted from a high-resource language, can be reused to steer behavior in low-resource languages without additional per-language data collection.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17880
PDF
-
Socratic Skill Learning: Interleaving Teaching and Practice for Transferable Agent Skills
2026-10-13 07:23:34Letian Zhu, Han Li, Yifan Yao, Xinping Lei, Xinyu Che, Jiahao Wang, Dianyi Wang, Yucheng Fu, Jiakai Wang, Zhiqi Bai, Wenbo Su, Jiaheng LiuAbstract
Agent Skills improve LLM agents by making task knowledge reusable across sessions. Learning transferable Skills requires evidence about a procedure and its conditions of use, which a successful task attempt can leave untested. We introduce Socratic Skill Learning (SSL), which interleaves expert teaching with Student practice. An Educator guides investigation while the Student remains the sole environment operator and Skill author, able to question, test, and adapt advice. To test reuse beyond the learning task, we construct SkillTransferBench, with 35 task families and 408 executable cases, pairing public variation specifications with held-out evaluation in fresh solver sessions. On the main 22-family, 155-case cohort, Fixed-Window SSL reaches 0.801 utility, versus 0.723 without a Skill, 0.761 for adapted Trace2Skill (trajectory distillation), and 0.751 for adapted SKILL-KD (contrastive distillation). Across three model pairs on shared families, interleaved guidance outperforms expert authoring and post-hoc teaching. Process analysis links guidance to Student writing and package revision; a case study shows the Student testing and refining a proposed procedure for later reuse. As demand for new lessons falls, the cost of fixed-window review grows. Issue-Guided SSL tracks unresolved questions across Skill revisions to select and focus review, reaching 0.805 utility with 61.7% fewer Educator tokens and 16.9% fewer total acquisition tokens than Fixed-Window SSL on the main cohort.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17860
PDF
-
Agentic-MatriBO: Capability-Compiled Agentic Bayesian Optimization for Materials and Chemical Design
2026-10-13 06:50:13Yu Zeng, Yuchuan Chen, Ziyue Zhang, Zijian Wei, Yanyi Su, Guojiang Zhao, Hongshuai Wang, Zhifeng GaoAbstract
Materials and chemical optimization must choose the next synthesis, characterization, or high-fidelity simulation from sequential observations under a limited budget. Large language models can use task knowledge to propose adjustments, but composing surrogates, acquisition functions, and search strategies is constrained by compatibility, parameter dependencies, and evidence requirements. We present Agentic-MatriBO, a capability-compiled agentic Bayesian optimization (BO) framework that represents BO components and selection conditions as composable capabilities. A language policy proposes BO recipes and candidate plans; a compiler checks proposals against the registered capability catalog; and a deterministic execution layer validates and commits candidates. Oracle observations are stored in research memory and inform later rounds. We compare Agentic-MatriBO with eight baselines on four offline finite candidate pools---COF, PCE10, Buchwald--Hartwig (BH), and Suzuki--Miyaura---using five seeds per condition; results are reported descriptively. Across the four tasks, Agentic-MatriBO has the best mean endpoint quality after orienting each objective so that higher is better. On COF, it reaches the tabulated optimum in all five seeds both in the end-to-end comparison and after shared initialization, where it also retains an advantage in cumulative quality. Its normalized area under the best-so-far quality curve during the BO stage (nAUC-BO) is substantially higher on COF, near parity with the best baseline on PCE10 and BH, and lower on Suzuki. COF ablations show that removing the capability compiler or evidence gate coincides with lower cumulative quality and fewer exact-optimum hits. The framework provides an inspectable interface for evidence-conditioned BO adaptation.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17832
PDF
-
Learning from fracture: Teaching language models to design through physical experience
Abstract
How can language models learn to optimize physical designs through experience? We introduce FracLearn, an experience-abstraction-generalization framework evaluated through a deliberately unfamiliar, Tetris-inspired fracture game. The model first proposes architectures and examines simulated fracture responses to accumulate experience. Abstraction organizes these observations into conditional principles linking reinforcement placement, crack evolution and load transfer, with supporting evidence and failure conditions. Generalization challenges the model to combine these principles in new architectures and surpass the earlier design record. Campaigns demonstrate progressive refinement and recovery through hypothesis revision, yielding a best composite score 11.0% above the experience-stage record. Fracture analyses connect improved performance to crack deflection and temporary trapping as toughening mechanisms, alongside greater residual load capacity. Learning occurs through external knowledge and decisions, without model-weight updates. This approach offers a route to scientific agents that turn computational experiments into reusable physical knowledge for materials discovery and engineering design.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17804
PDF
-
Keep Your Receipts: Trustworthy Cleanup of Agent-Built Knowledge Graphs
Abstract
Agents build knowledge graphs as memory, and building them has become easy. Maintaining them has not. When a source is retracted, overruled, or erased, everything built on top of it (entity merges, entailed edges, community summaries) must be re-examined, and the maintainer must choose between deleting too much and too little. Deployed pipelines delete only the directly extracted facts: on our corpus that reduces contaminated queries from 32.2% to 28.0% under a real retraction and changes nothing under predicate revocation. Rebuilding is no repair for a constructor that samples: two builds of the same 300-document corpus differ in 29,475 artifacts, five times the 6,248 that removing the 61 retracted documents changes. PolyGraph instead asks the constructor for receipts: for every artifact, alternative bundles of source evidence, each claimed sufficient on its own, emitted in the call that creates it. Cleanup is then set arithmetic over the receipts, with no model calls. Because receipts are the model's own self-report, a re-derivation audit checks them by deleting the cited evidence and testing whether the constructor still produces the artifact. The audit finds that a receipt is trustworthy when it is shaped like the claim it backs. Merge receipts, each backing one narrow claim, agree with re-derivation 68% and 85% of the time. Summary receipts, elicited per sentence but stored as one flat list for the whole text, are wrong about two thirds of the time, almost always by keeping dead content. One receipt per sentence instead is 100% and 88% faithful and lets cleanup keep 33 of 35 touched summaries. The result holds under a second oracle model, a claim-blind prompt, ten further deletions, a second domain and a second constructor. Distrusting the summary layer cuts contaminated queries to 0.4% and 0.8% at 99.8 to 100% of the recall a perfect maintainer would return.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17719
PDF
-
Multimodal Contrastive Representation Learning Based Voice Conversion for Electrolaryngeal Speech Enhancement
2026-10-13 04:47:07Ding Ma, Jinyi Mi, Jiajun He, Kazuhiro Kobayashi, Naoki Nishio, Tomoki TodaAbstract
Electrolaryngeal (EL) speech suffers from severe acoustic mismatches with normal speech, resulting in degraded naturalness and intelligibility. These mismatches also make sequence-to-sequence voice conversion struggle to learn robust intermediate representations, hindering EL-speech-to-normal-speech conversion (EL2SP) performance. To address this, we propose a novel multimodal contrastive representation learning framework to 1) achieve richer multimodal-representation learning, and 2) facilitate the transferability of multimodal representations to speech-only EL2SP. First, EL speech and text representations are fused to learn multimodal representations for target speech generation, during which cross-modal contrastive learning is employed to encourage a semantically consistent speech--text representation space. Next, the fused representations are treated as a fixed target to guide speech-only representation learning through cross-stage contrastive learning, enabling multimodal knowledge to be transferred to the EL2SP task while preserving architectural and inference simplicity. Experimental results demonstrate that the proposed method consistently outperforms both the speech-only and multimodal-based baselines.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17716
PDF
-
A Node Embedding Framework for Model-Centric Conformance Checking in Process Mining
2026-10-13 04:34:39Mostafa Abbasi, Homayoun NajjaranAbstract
Evaluating the quality of process models discovered from event logs is a fundamental challenge in process mining. Traditional evaluation relies on conformance metrics such as fitness and precision, which are highly sensitive to noise and deviations in event logs and therefore often fail to give a consistent assessment of discovery algorithms. We introduce a model-centric evaluation framework that combines node embeddings with Procrustes alignment to enable cross-graph comparability, since embeddings derived from different graphs reside in distinct vector spaces. The proposed PetriNetGraph method transforms Petri net process models into graph representations compatible with embedding algorithms. Comparing the node embeddings of ground-truth and discovered models through a parameterized cosine distance yields a unified and bounded metric for evaluating process discovery across datasets and methods. We evaluate the framework under static and context-aware scenarios to examine how prior knowledge of activity existence influences the assessment of structural similarity. The framework also integrates a compliance checking mechanism that localizes deviations at the activity level and an entropy-regularized node distance aggregation function that quantifies global deviations across the model. Experiments on nine publicly available event logs show that the approach provides a reliable embedding-based conformance indicator that reflects the behavior of fitness and precision. Experiments on synthetic logs with controlled noise levels confirm that it is substantially more robust to log deviations than traditional conformance metrics, PetriNet2Vec, and its dynamic-window and Procrustes-aligned variants, while remaining sensitive to process model degradation. All implementation details and datasets are publicly available.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17702
PDF
-
MedEvo: Medical Reasoning Evolution through Knowledge Navigation
2026-10-13 04:11:26Xiaoliang Wang, Zhuo Chen, Chuanyang Zheng, Quanbin Wang, Jiangwei Lao, Yingao Liu, Jiajia Liu, Jinming Cao, Honglin Lin, Zhihong Lu, Hao Jin, Jian Wang, Lijun WuAbstract
Medical multimodal reasoning models are typically trained on teacher-generated rationales, discarding the valid reasoning in students' own failed trajectories, while general-domain correction methods offer little guidance for errors in image reading, anatomical context, or differential diagnosis. We propose MedEvo, which uses medical knowledge, namely a tag taxonomy and error-type diagnosis, to navigate both trajectory repair and data selection. For each failed rollout, a gold-blind auditor locates the first consequential error and provides a minimal, answer-free hint; the same student retains the valid prefix and regenerates the erroneous suffix. After each round, MedEvo identifies capability gaps across medical tags on a held-out feedback set and prioritizes related questions for new rollouts and repairs. With supervised fine-tuning (SFT) alone, MedEvo raises Qwen3.5-9B mean accuracy across seven benchmarks from 65.57% to 69.12% (47.40% to 54.50% on the multimodal track of MedXpertQA (MedXpertQA-MM)), surpassing teacher-distillation SFT (66.93%). It also raises Qwen3.5-35B and Qwen3.5-397B mean accuracy from 67.91% to 73.04% and from 71.83% to 74.95%, respectively. Our analyses show that failed trajectories contain reasoning worth preserving, that repair success does not imply learning value, and that independent feedback is key to sustained improvement.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17690
PDF
-
Can Agents Truly Forget? Explore Machine Unlearning for LLM Agents
2026-10-13 03:37:30Junkai Lin, Ge Xu, Zheng Xu, Junkai Chen, Xiangyuan LanAbstract
Large language model (LLM) agents have shown strong capabilities in interactive problem solving, tool use, and long-horizon task execution. However, they may also acquire sensitive or private knowledge during training and expose it during deployment. Retraining a model from scratch to remove such knowledge is prohibitively costly, making machine unlearning a more practical approach. Machine unlearning aims to remove targeted undesirable knowledge from trained models without retraining from scratch while preserving general utility. When language models are deployed as agents, parameter-level unlearning becomes insufficient: forgotten information may still persist in memory artifacts or be reacquired via tool calls. In this paper, we formulate the problem of agent unlearning as removing private information from an LLM-based agent across three information channels: parametric knowledge, persistent memory, and tool-call acquisition. To study this problem, we introduce we introduce Agent Unlearning Benchmark (AUBench), a controlled benchmark that extends the fictitious-author setting of TOFU into an agent environment with persistent memory, an offline web snapshot, tool-accessible documents, and provenance-tracked interaction trajectories. By evaluating existing parameter-level and memory-level unlearning methods in this setting, AUBench reveals the gap between model-level forgetting and system-level agent unlearning, and provides a foundation for developing unlearning methods that operate across the full lifecycle of LLM agents.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17659
PDF
-
MAS-Gate: Unlocking Multi-Agent System Lifecycle via Node-Centric Quantification
2026-10-13 02:33:28Hehai Lin, Zixuan Wang, Ming Yan, Peidong Wang, Shilei Cao, Minzhi Li, Bosheng Ding, Juepeng Zheng, Chengwei QinAbstract
Large language model (LLM)-based multi-agent systems (MAS) can solve complex problems beyond the reach of a single agent by decomposing tasks and coordinating complementary capabilities. Despite extensive research on MAS, existing work largely focuses on specific aspects or individual phases of their development and operation, resulting in fragmented perspectives with objectives and signals that are difficult to reconcile across these different settings. To bridge these disconnected perspectives, we conceptualize MAS development and operation as a three-phase lifecycle spanning design-time orchestration before execution, runtime intervention during execution, and post-hoc attribution after execution, and introduce MAS-Gate, a unified node-centric quantification applicable across these phases. Modeling each execution as a directed acyclic graph, MAS-Gate combines within-node utility, which estimates a node's contribution through changes in reference-answer likelihood, with inter-node strength, which approximates semantic uptake and potential downstream reach. We adapt this shared quantification to three objectives: providing a structure-aware dense reward for Orchestrator training, training an Editor to act on execution prefixes using paired rewards from completed edited and unedited trajectories, and supplying risk-ranked nodes as auxiliary evidence for LLM-based post-hoc attribution. Experiments spanning interaction environments, knowledge-intensive multi-hop question answering, general reasoning, and post-hoc attribution demonstrate that MAS-Gate improves workflow construction, targeted runtime editing, and LLM-assisted error localization over corresponding phase-specific baselines. Together, these results establish MAS-Gate as a unified structural interface for improving MAS generation, dynamic editing, and error attribution throughout the lifecycle.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17613
PDF
-
Tanawpamana: Benchmarking Vision-Language Models on Philippine Cultural Heritage Sites
2026-10-13 02:15:52Ivan Yuri De Leon, Hiroki Ouchi, Sakriani SaktiAbstract
Vision-language models are increasingly used to identify and describe places, yet their recognition and knowledge of specific cultures remain poorly understood, particularly in underrepresented regions such as the Philippines. This study introduces the Tanawpamana Dataset, a repository of Philippine heritage site images with metadata based on the Talapamana registry of the National Commission for Culture and the Arts of the Philippines. We also introduce the Tanawpamana Benchmark, comprising 4,046 tasks designed to evaluate models' recognition and knowledge of heritage sites. Evaluating seven vision-language models, we find that they struggle more with fine-grained visual discrimination among similar heritage sites than with coarse recognition, and that their geographic knowledge is weaker when inferred from image inputs than from text.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17596
PDF
-
Large Language Models Extend Known Research But Do Not Create New Research Directions
2026-10-12 21:57:24Zhihua Liu, Kurt Butler, Damian Machlanski, Danilo Bueno, Sotirios A. TsaftarisAbstract
The success of autonomous research systems relies on the quality of the scientific hypotheses they generate. Recent studies report that human reviewers often rate LLM-generated ideas as more novel than those from human experts, raising hope that AI can accelerate scientific discovery. Yet it is unclear whether these models can do more than extend existing knowledge. Can LLMs drive genuine innovation by revealing new research directions, or are they limited to repeating and recombining what science already knows? This unresolved question lies at the heart of discussions about AI and the future of science. To address this, we distinguish two types of novelty: distance, which measures how far a hypothesis lies from existing work, and direction, which captures whether the hypothesis pursues new lines of inquiry beyond those already present. These two measures diverge sharply for LLM-generated work. We represent scientific papers in a shared conceptual space using a pretrained scientific-paper embedding model. Within this space, we embed 26.9 million human-authored papers across 1,325 research topics in six natural-science fields and 524,308 LLM-generated hypotheses from eight models. Distance is measured using existing metrics. To quantify direction, we introduce the orthogonal novelty score (ONS). LLMs consistently generate ideas more distant from existing work than human-authored papers, up to 16\% further when the LLM is given the papers farthest from the centre of the topic. However, LLM hypotheses have a 15--23\% lower median ONS than published papers from the same topic. Thus, LLMs move further from what is already known, but remain within established directions. This pattern persists across changes in various configurations. As a result, LLMs excel at extending the frontier but the creation of new directions is still led by human scientists.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17439
PDF
-
Why Agentic Workflows Fail: A Taxonomy Derived from Literature and Empirical Data
2026-10-12 21:32:14Xin Li, Rami Bahsoon, Tao ChenAbstract
Agentic workflows combine large language models with planning, memory, tools, execution environments, and iterative feedback. Their failures are difficult to diagnose because the same user-visible symptom may originate from model behavior, workflow orchestration, external tools, infrastructure, or their interaction. This paper presents a source-aware taxonomy of agentic workflow failures and applies it to evidence from academic literature and GitHub. The taxonomy combines source attribution with three empirical dimensions: a concrete failure-type dictionary, workflow-stage localization, and agent-capability localization. Its central structure is a two-axis localization matrix: workflow stages form the horizontal process axis and agent capabilities form the vertical functional axis. This design moves beyond taxonomies that mainly classify final failure manifestations by localizing intermediate process states inside agent operation. Our empirical comparison shows that literature-derived failures emphasize planning, reasoning, learning, perception, and knowledge-related mechanisms, whereas GitHub-derived failures concentrate on execution-stage, actuation-related, tool, API, and infrastructure failures. The dominant hotspot is iterative execution and actuation. These findings indicate that agentic workflow reliability should be treated as a systems problem rather than only a model-output problem. The taxonomy provides a basis for structured evaluation, root cause analysis, failure injection, and engineering triage.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17418
PDF
-
HASTE: Hierarchical and Attribute-aware Supervision for Generative Recommendation
2026-10-12 20:25:50Tripti Shukla, Shwetha Somasundaram, Akshat Gupta, Rohit Garg, Paul Duff, Ian AndersonAbstract
Generative recommendation models have shown promising results by representing items as semantic IDs and generating these IDs autoregressively. We introduce an approach showing that explicitly incorporating hierarchical and attribute-level supervision into generative recommendation can simultaneously improve training efficiency and long-tail recommendation performance. Existing approaches typically optimize semantic ID prediction with a standard causal cross-entropy objective, which does not explicitly exploit the hierarchical structure of semantic IDs and can require substantial training to learn effective item representations. Moreover, data-sparse tail items often suffer from poor representations due to limited supervision, resulting in weaker long-tail recommendation performance. We address these challenges with HASTE (Hierarchical and Attribute-aware Supervision for generaTive rEcommendation), a unified training framework that introduces hierarchical and attribute-specific supervision. HASTE's Hierarchical Loss explicitly exploits the multi-level structure of semantic IDs, providing supervision at each level of the item hierarchy and substantially accelerating training convergence. Its Attribute-Specific Losses encourage the model to preserve fine-grained item attributes, leveraging the language model's inherent semantic knowledge to improve representations and recommendations for tail items. We extensively evaluate HASTE on a fashion recommendation benchmark and across multiple domains of the Amazon dataset. HASTE improves recommendation performance across datasets and item-frequency regimes. Notably, the Hierarchical Loss achieves approximately 2x faster convergence than the standard causal cross-entropy objective, substantially improving training efficiency while maintaining strong recommendation quality.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17368
PDF
-
Physics: Benchmarking Foundation Models on University-Level Physics Problem Solving
2026-10-12 18:17:30Kaiyue Feng, Yilun Zhao, Yixin Liu, Tianyu Yang, Chen Zhao, John Sous, Arman CohanAbstract
We introduce Physics, a benchmark for evaluating foundation models on advanced physics problem solving. It contains 1034 annotated, open-ended problems from PhD-qualifying exams across six core physics domains. Solving these problems requires applying domain-specific knowledge to physical setups and carrying out multi-step mathematical reasoning. Each problem includes a worked solution and independently checked reference answers. An LLM-based grading protocol evaluates individual answers and complete multi-part problems, supporting numeric, symbolic, and text responses. We evaluate 53 proprietary and open-weight models. On test problems without figures, high accuracy is concentrated in a small group of thinking models, with the strongest open-weight models approaching proprietary performance. Most open-weight models still solve fewer than half of these problems. Our analysis of Qwen3.5-9B reveals failures in applying physics knowledge and interpreting figures, with misread figures affecting several subquestions.
Abstract (translated)
URL
https://arxiv.org/abs/2503.21821
PDF
-
From Conceptual EneMonIE to an Executable Backend: A Modular Architecture for Energy Information Extractionand Grounded Question Answering
2026-10-12 18:09:35Janik Lesage, Khadim L\^o, and Jean-S\'ebastien DessureaultAbstract
The original EneMonIE proposal describes a Web-based semantic system for continuously monitoring energy trends through targeted information extraction from heterogeneous online media. This article presents EneMonIE-Backend, an implementation-derived redesign that converts that conceptual pipeline into a modular, executable, and application-independent backend. The architecture separates five concerns: data sources, acquisition and normalization, semantic processing, knowledge organization, and persistent storage. It operationalizes configurable connectors for RSS feeds, URL collections, arXiv, social-media JSON files, and open-data CSV files; it also introduces question-driven Web search and scraping. Relevant documents pass through energy categorization, named entity recognition, temporal and event extraction, source sensitive sentiment analysis, and extractive summarization. The resulting semantic representations support entity co-occurrence graphs, cross-document interlinking, cluster-level information fusion, and relational persistence in SQLite. Three grounded question-answering paths are exposed at the backend level: live Web evidence, the stored corpus, and selected configured resources. Answer generation uses a language model when configured and otherwise falls back to deterministic extractive output. Unlike the original proposal, the presented architecture deliberately excludes the user-interface and server-side application layers, allowing the semantic core to be embedded in different applications. A deterministic smoke validation confirms relevance filtering, persistence, clustering, evidence retrieval, and document deletion. The work provides a reproducible architectural basis for future evaluation on larger multilingual energy corpora.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17243
PDF
-
Deleting Code Without Losing Knowledge: Regenerative Software for Python Libraries
Abstract
In the age of "Software 3.0", when a complete program can be rewritten in minutes by an agent, the traditional approach of relying on code as the sole source of truth becomes questionable. Fowler advocated for a set of seven primitives, which lay the framework for regenerative software, so that programs can be deleted and rebuilt without losing their knowledge. The knowledge lives in machine-readable records in a tracked directory, from which a *regeneration bundle* is built for each unit. The regeneration bundle is a single file with the module's architecture, intent, decisions, public contract, evaluations, and consumers, but no implementation, plus the one dependency module it may import. The open-source package spotregen implements these primitives and provides a graphical user interface. This report presents a first application of Fowler's primitives to the safety-critical forecasting library spotforecast2-safe. During the first experiments, the Claude Code session that orchestrated the study consumed more than 43 million tokens, almost all of them its own history re-sent with every request. This motivated the switch to an architecture that is built on local models. Deterministic checks and the maintainer judged the results. The report contains executable examples for every primitive. Each example runs spotregen against the repository or against a throw-away copy of it.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17197
PDF
-
ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning
2026-10-12 17:53:56Aotian Chen, Frazier N. Baker, Reza Averly, Shan Zhang, Tianao Chen, Daniel Adu-Ampratwum, Xia Ning, Yuan XueAbstract
Current chemistry benchmarks primarily evaluate outcome prediction or reaction pathway generation, but rarely test whether a model can explain why an observed chemical effect occurs. We introduce ChemExplain, a benchmark of 1,180 multiple-choice chemistry questions drawn from 383 peer-reviewed papers. Each question pairs a molecular or reaction image with an observed chemical effect and four candidate explanations, each associated with a different structural feature visible in the image. Answering requires the model to perceive the depicted structure, apply chemical knowledge, and identify which feature most directly explains the effect, a capability we term explanatory attribution. The correct feature-mechanism relation is derived from supporting evidence in the source paper rather than invented by the question-generation LLM. We construct the benchmark automatically with a multi-agent framework that applies progressive quality control at three stages and validate it with eight chemistry domain experts. The strongest evaluated model reaches 78.6% accuracy, leaving considerable room for improvement. On a 600-question expert-reviewed subset, reasoning-trace analysis classifies 71.4% of the strongest model's errors as attribution failures. Attribution is also the largest error category for the open-source and chemistry-specific models included in our analysis.
Abstract (translated)
URL
https://arxiv.org/abs/2610.17183
PDF
-
Toward Fine-Grained Forgetting:Attribute Unlearning for Multimodal Large Language Models
2026-10-12 16:44:31Junkai Lin, Junkai Chen, Siqi Hou, Yuhao He, Ruiqi Liu, Chenhan Jin, Shengze Xu, Tieyong Zeng, Xiangyuan LanAbstract
Multimodal large language models (MLLMs) can memorize and disclose sensitive information, motivating machine unlearning to remove designated knowledge while preserving model utility. Existing privacy-oriented benchmarks mainly study profile-level deletion, whereas practical requests often require finer-grained forgetting: removing a sensitive attribute while retaining other information about the same identity. We introduce attribute-level MLLM unlearning and construct AMU-Bench, covering long-text, numeric, and short-text targets, multiple forget ratios, and diverse question types. Our evaluation shows that target and retain attributes share identity-specific and visual evidence, making selective forgetting prone to residual leakage and collateral degradation. To address this challenge, we propose Causal Localization and Retain-Aware Projection (CLRP), a lightweight, training-free framework that localizes causally relevant layers and projects intermediate representations away from target-specific subspaces while preserving retain-related information. Experiments across MLLMs with different architectures and scales show that CLRP achieves a strong forgetting--retention trade-off on AMU-Bench. For example, at a 5% forget ratio, CLRP reduces Attribute Forget Set cloze accuracy from 20.00% to 4.00% while improving Attribute Retain Set accuracy from 14.77% to 17.79% in the reported LLaVA-1.5-7B and Qwen2.5-VL-3B results. These findings demonstrate effective and selective attribute-level unlearning without modifying model parameters.
Abstract (translated)
URL
https://arxiv.org/abs/2608.01008
PDF
-
AUGUstus: Autonomous Driving VLA Model with Unified Generation and Understanding
2026-10-12 15:42:09Yuqi Ye, Caigao Jiang, Zhuonan Wang, Congyuan Pei, Chao Qu, Changhao Peng, Junhong Lin, Qize Yu, Zetian Song, Guangzhao Fu, Chongruo Wu, Shiyu Huang, Shuchang Zhou, Honggou Yang, Wei GaoAbstract
Vision-language-action (VLA) models show strong potential for end-to-end autonomous driving by leveraging world knowledge and understanding to excel in long-tail scenarios. However, current VLAs have two limitations: 1) Degeneration into vision-action (VA) models. For low latency, regression/flow-based methods directly generate actions but underuse VLM understanding, causing unsafe, unexplainable actions. 2) Autoregressive high latency. Autoregressive methods sequentially generate chain-of-thought (CoT) and action tokens, fully leveraging VLM understanding, but high latency hinders deployment. To enable low-latency deployment while using CoT to assist action generation, we propose AUGUstus, an Autonomous-driving VLA model with Unified Generation and Understanding. We unify diffusion-based action generation and autoregressive CoT understanding in one model, where understanding only serves generation. Joint CoT supervision preserves VLM spatial understanding in shared representation for safer, more accurate trajectories. AUGUstus-mode orders action generation before CoT, so CoT supervises reliable actions during training while inference needs only action output, enabling low latency. AUGUstus-GRPO, a unified GRPO algorithm, jointly optimizes action and CoT, improving action performance. Experiments show AUGUstus achieves 94.6 PDMS on NAVSIMv1 and 90.7 EPDMS on NAVSIMv2, SOTA planning without scorer module. On nuScenes, it achieves average L2 error 0.27 m and average collision rate 0.05%, best among compared methods. To validate AUGUstus practicality, we train on in-house data with 512 NVIDIA H800 GPUs for ~2 days, evaluate on an internal 1,000 km dataset, and preliminarily deploy on a real vehicle, achieving 12 Hz operation. Project page: this https URL
Abstract (translated)
URL
https://arxiv.org/abs/2610.16985
PDF
