Tag
Group Relative Policy Optimization
Posts and notes tagged with Group Relative Policy Optimization.
- August 16, 202612 min read
Dr. GRPO: Group Relative Policy Optimization Done Right
How Dr. GRPO fixes two biases in GRPO: dividing by the response length makes long wrong answers cheap, and dividing by the group's standard deviation overweights questions that are too easy or too hard. Removing both recovers the unbiased policy gradient and stops responses from growing needlessly long.
- June 10, 202618 min read
Group Relative Policy Optimization (GRPO)
How Group Relative Policy Optimization (GRPO) removes PPO's value model by comparing answers within a group, and why it works so well for training reasoning models. Part 6 of a series on policy optimization for LLMs.