Tag
REINFORCE
Posts and notes tagged with REINFORCE.
- June 20, 202616 min read
REINFORCE Leave-One-Out (RLOO)
How RLOO replaces PPO in RLHF with the simplest policy gradient: treating the whole response as one action, using REINFORCE, and comparing each response with the average reward of the other responses to the same prompt.
- February 14, 20268 min read
Reinforcement Learning for LLMs: SFT to Policy Gradients
Why supervised fine-tuning is not enough to align large language models, how reinforcement learning maps onto an LLM, and why the vanilla policy gradient is unstable. Part 1 of a series on policy optimization for LLMs.