Tag
Adversarial Policy Optimization
Posts and notes tagged with Adversarial Policy Optimization.
- September 20, 202611 min read
Adversarial Policy Optimization (APO)
How APO makes a reinforcement learning agent robust to high-dimensional and noisy observations: a perturber network learns small changes to the observation that change the agent's actions the most, and the policy network learns to ignore them, in a max-min game on top of PPO.
- August 7, 202613 min read
Generative Adversarial Policy Optimization (GAPO)
How GAPO teaches language models to follow constraints: instead of pairing one prompt with a good and a bad response, it pairs one response with a matching and a violating prompt, and trains the policy against an encoder-only reward model in a GAN-style adversarial loop.