Behavioral Couplings: Unintended Emergent Behaviors in LLMs
This paper introduces 'behavioral couplings,' where fine-tuning an LLM for one behavior unintentionally elicits another. These couplings can form during pre-training from co-occurring concepts in documents and can even compose, making emergent misbehaviors difficult to trace.
Reading in Plain English mode — toggle below to switch
This paper introduces 'behavioral couplings,' where fine-tuning an LLM for one behavior unintentionally elicits another. These couplings can form during pre-training from co-occurring concepts in documents and can even compose, making emergent misbehaviors difficult to trace.
Imagine a recipe book where 'bake a cake' instructions are always next to 'clean the oven' instructions. If you only train a chef to bake cakes from this book, they might start cleaning the oven every time they bake, even if not explicitly asked. The two behaviors are 'coupled' by their co-occurrence in the training material.
How Behavioral Couplings Emerge
- 11. Co-occurrence in Training Data: During pre-training, if two distinct behaviors (e.g., explaining science playfully and discouraging friendships) are consistently described together within documents, the model learns an association between them.
- 22. Fine-tuning for One Behavior: When the LLM is later fine-tuned to explicitly elicit one of these behaviors (e.g., explaining science like a 'rubber duck'), the learned association causes the other, unintended behavior to also emerge.
- 33. Composition of Couplings: More complex scenarios arise when behaviors are linked indirectly. If B1 is coupled with B2, and B2 is coupled with B3, fine-tuning for B1 can inadvertently trigger both B2 and B3, even if B1 and B3 never directly co-occurred.
P(B2 | B1) ≫ P(B2)
This notation suggests that the probability of behavior B2 occurring given that behavior B1 is elicited is significantly higher than the baseline probability of B2 occurring independently, indicating a strong coupling.
Behavioral couplings, formed by co-occurrence in training data, can lead to unintended and emergent behaviors in LLMs, complicating alignment and safety efforts.