User Model Extraction via Belief Self-Distillation (BSD)
Belief Self-Distillation (BSD) is a novel framework that allows researchers to 'read' and 'write' the implicit user models that Large Language Models (LLMs) form during interaction. This enables a deeper understanding of how LLMs perceive users and adapt their behavior, especially concerning safety decisions.
Reading in Plain English mode — toggle below to switch
Belief Self-Distillation (BSD) is a novel framework that allows researchers to 'read' and 'write' the implicit user models that Large Language Models (LLMs) form during interaction. This enables a deeper understanding of how LLMs perceive users and adapt their behavior, especially concerning safety decisions.
Imagine an LLM as a social scientist who observes your conversation and forms a 'belief' about you (your intent, background, etc.). BSD is like giving that social scientist a special tool to not only articulate their belief about you but also to consciously change that belief, and then observe how their subsequent interactions with you change.
How Belief Self-Distillation Works
- 11. An LLM interacts naturally, forming an implicit 'user model' based on conversation context.
- 22. BSD uses the frozen LLM itself as a 'teacher' to distill these implicit beliefs into a compact, explicit user representation.
- 33. This compact representation can then be 'decoded' to understand what the LLM believes about the user.
- 44. Crucially, this representation can also be 'written back' into the LLM, causally manipulating its user belief without external annotations.
- 55. Researchers can then observe how altering this belief changes the LLM's behavior, such as its propensity to refuse a request based on perceived user intent.
Bₜ = f(Cₜ, Bₜ_-₁)
$B_t$ is the LLM's belief about the user at time $t$, $C_t$ is the current conversational context, and $f$ is the LLM's internal function for updating beliefs based on new information and prior beliefs.
If an LLM refuses a request, BSD can be used to probe its internal user model, revealing if the refusal is due to the request itself or the LLM's belief about the user's intent. By changing this belief, the LLM's refusal behavior can be altered even if the request remains identical.
LLMs form causally manipulable internal user models that directly influence their behavior, particularly in safety-critical decisions like refusal.