Tag
Trust Region Policy Optimization
Posts and notes tagged with Trust Region Policy Optimization.
- March 1, 202638 min read
Trust Region Policy Optimization (TRPO)
How Trust Region Policy Optimization (TRPO) keeps policy updates stable by limiting how much a model's behavior can change, explained step by step with KL divergence, importance sampling, the Fisher information matrix, and the natural gradient. Part 2 of a series on policy optimization for LLMs.