GAPO

Generative Adversarial Policy Optimization (GAPO)

How GAPO teaches language models to follow constraints: instead of pairing one prompt with a good and a bad response, it pairs one response with a matching and a violating prompt, and trains the policy against an encoder-only reward model in a GAN-style adversarial loop.