Abstract
Deep learning models for drug-like molecules and proteins overwhelmingly reuse transformer architectures designed for natural language, yet whether molecular sequences benefit from different designs has not been systematically tested. We deploy autonomous architecture search via an agent across three sequence types (SMILES, protein, and English text as control), running 3,106 experiments on a single GPU. For SMILES, architecture search is counterproductive: tuning learning rates and schedules alone outperforms the full search (p = 0.001). For natural language, architecture changes drive 81% of improvement (p = 0.009). Proteins fall between the two. Surprisingly, although the agent discovers distinct architectures per domain (p = 0.004), every innovation transfers across all three domains with <1% degradation, indicating that the differences reflect search-path dependence rather than fundamental biological requirements. We release a decision framework and open-source toolkit for molecular modeling teams to choose between autonomous architecture search and simple hyperparameter tuning.
Abstract (translated)
针对类药分子和蛋白质的深度学习模型，绝大多数直接复用为自然语言设计的Transformer架构，但分子序列是否能从不同架构中获益，尚缺乏系统性验证。我们通过智能体在三种序列类型（SMILES、蛋白质及作为对照的英文文本）上开展自主架构搜索，在单张GPU上运行了3,106次实验。对于SMILES，架构搜索适得其反：仅调整学习率及其调度策略就已优于完整搜索（p = 0.001）。对于自然语言，架构改进贡献了81%的性能提升（p = 0.009）。蛋白质则介于两者之间。令人惊讶的是，尽管智能体在各领域发现了不同的架构（p = 0.004），但每个创新都能跨所有领域迁移，性能损失不足1%，这表明差异源于搜索路径依赖，而非根本性的生物学需求。我们发布了一套决策框架及开源工具包，帮助分子建模团队在自主架构搜索与简单超参数调优之间做出选择。
URL
https://arxiv.org/abs/2603.28015
