Abstract
In visually ambiguous manipulation such as detecting button click tactile feedback is often the sole source of ground truth. However, fusing tactile data poses a significant challenge due to a spatiotemporal mismatch: tactile perception requires high-frequency processing with long-horizon memory (System 1), whereas visual policies operate at low control frequencies (System 2). Existing architectures struggle to bridge this gap: Transformers are computationally prohibitive for high-frequency loops (>100Hz), while LSTMs suffer from forgetting over extended interaction histories. In this paper, we introduce TacMamba, a hierarchical architecture that aligns high-bandwidth tactile reflexes with low-frequency visual planning. Our approach comprises three core contributions: (1) a custom high-frequency tactile interface designed for flexible integration; (2) a Mamba-based Tactile History Compressor that encodes continuous force history into a compact state with O(1) inference latency (0.45 ms), enabling plug-and-play fusion with VLA models without joint pre-training and (3) a Tactile-Guided Dual-Stage Training strategy that leverages temporal discrimination for self-supervised representation learning and phase-uniform sampling to mitigate data sparsity. Experiments on discrete counting and implicit state switching demonstrate that TacMamba achieves 100% success rates, significantly outperforming the visual-only pi_0.5 baseline, while strictly satisfying hard real-time constraints.
Abstract (translated)
在视觉模棱两可的操作中，如检测按钮点击时，触觉反馈通常是唯一的真实来源。然而，融合触觉数据面临着显著挑战，因为存在时空不匹配的问题：触觉感知需要高频处理和长时间记忆（系统1），而视觉策略则以低控制频率运行（系统2）。现有的架构难以弥合这一差距：Transformer对于高频率循环（>100Hz）来说计算成本过高，而LSTM在长交互历史中会遗忘信息。 在这篇论文中，我们引入了TacMamba，这是一种层次化架构，将高频触觉反射与低频视觉规划对齐。我们的方法包含三个核心贡献： 1. 设计了一种定制的高频率触觉接口，以实现灵活集成； 2. 一个基于Mamba的触觉历史压缩器，它能将连续力的历史编码为具有O(1)推理延迟（0.45毫秒）的小型状态，使得无需联合预训练即可与视觉语言动作(VLA)模型无缝融合； 3. 触觉引导的双阶段训练策略，利用时间区分进行自监督表示学习，并采用相位均匀采样来缓解数据稀疏性。 在离散计数和隐式状态切换实验中显示，TacMamba能够实现100%的成功率，显著优于仅基于视觉的pi_0.5基准模型，同时严格满足硬实时约束。
URL
https://arxiv.org/abs/2603.01700
