Temporal Gradient Inversion for Private Trajectory Reconstruction
This paper introduces TRACE, a novel attack that reconstructs private observation-action trajectories from policy gradients in distributed embodied reinforcement learning. It exploits temporal correlations and policy-head gradient structure, outperforming prior methods in speed and accuracy. This highlights a critical privacy vulnerability in distributed RL systems.
Reading in Plain English mode — toggle below to switch
This paper introduces TRACE, a novel attack that reconstructs private observation-action trajectories from policy gradients in distributed embodied reinforcement learning. It exploits temporal correlations and policy-head gradient structure, outperforming prior methods in speed and accuracy. This highlights a critical privacy vulnerability in distributed RL systems.
Imagine you're trying to reconstruct a secret movie (the agent's trajectory) by only looking at the tiny, anonymized instructions sent to a remote editor for each frame (policy gradients). TRACE is like a super-smart detective who notices that the instructions for one frame are subtly related to the next, allowing them to piece together the entire sequence, even if individual instructions seem harmless.
How TRACE Reconstructs Private Trajectories
- 11. **Intercept Policy Gradients:** The attack intercepts the policy gradients that embodied agents send to a central server, which are usually considered private.
- 22. **Exploit Temporal Correlations:** TRACE analyzes the sequence of these gradients, recognizing that gradients from successive time steps are not independent but carry information about the agent's continuous movement.
- 33. **Recover Actions from Policy Head:** It uses the structure of the policy head's gradients to precisely recover the actions taken by the agent at each step.
- 44. **Autoregressive Reconstruction:** By combining the recovered actions with the temporal correlations, TRACE autoregressively reconstructs the full sequence of observations and actions, effectively revealing the agent's private trajectory.
I(Xₜ; Xₜ_+₁ | Gₜ, Gₜ_+₁)
This represents the information shared between two consecutive observations ($X_t$, $X_{t+1}$) given their respective gradients ($G_t$, $G_{t+1}$). TRACE exploits this hidden correlation.
In a robotic arm learning to pick up objects, the raw camera feed and joint movements are private. The robot sends only gradients to a central server. TRACE could reconstruct the exact sequence of camera views and joint angles, revealing the specific objects it interacted with and how it moved, even though only gradients were transmitted.
Even seemingly anonymized policy gradients can leak significant private information about an agent's trajectory, especially when temporal correlations are exploited.