Abstract
Activation steering methods control large language model (LLM) behavior by modifying internal activations at inference time. However, most existing activation steering methods rely on a fixed steering strength, leading to either insufficient control or unadapted intervention that degrades text plausibility and coherence. We introduce In-Distribution Steering (IDS), a novel method that adapts steering strength based on the input data distribution in representation space. IDS dynamically adjusts interventions according to how far a given input lies within the distribution, enabling adaptive intervention and generation stability during text generation. Experiments demonstrate that IDS achieves strong accuracy on classification tasks while producing coherent text without collapse, making IDS particularly well suited for real-world applications.
Abstract (translated)
激活导向方法通过在推理时修改大型语言模型（LLM）的内部激活来控制其行为。然而，大多数现有的激活导向方法依赖于固定的导向强度，这可能导致对模型行为的控制不足或过度干预，从而降低生成文本的真实性和连贯性。我们引入了一种新颖的方法——分布内导向（IDS），它根据输入数据在表示空间中的分布动态调整导向强度。IDS能够根据给定输入与训练数据分布的距离来调整干预措施，这使得在文本生成过程中可以实现自适应的干预和生成稳定性。 实验结果表明，IDS在分类任务中取得了优异的表现，并且在不牺牲连贯性的前提下保持了文本的稳定生成。因此，IDS特别适合于现实世界的应用场景。
URL
https://arxiv.org/abs/2510.13285
