Tag
Importance Sampling
Posts and notes tagged with Importance Sampling.
- September 13, 202611 min read
Group Sequence Policy Optimization (GSPO)
How GSPO fixes the instability of GRPO by moving the importance ratio from tokens to whole responses: one length-normalized sequence ratio, sequence-level clipping, and equal weights for every token, which stabilizes large Mixture-of-Experts training and powered Qwen3.
- September 3, 202610 min read
Clipped IS-weight Policy Optimization (CISPO)
How CISPO, from MiniMax-M1, keeps every token in the gradient: instead of clipping token updates like PPO and GRPO, which silently drops rare reasoning tokens such as 'Wait' and 'However', it clips the importance sampling weight and lets every token keep learning.
- March 1, 202638 min read
Trust Region Policy Optimization (TRPO)
How Trust Region Policy Optimization (TRPO) keeps policy updates stable by limiting how much a model's behavior can change, explained step by step with KL divergence, importance sampling, the Fisher information matrix, and the natural gradient. Part 2 of a series on policy optimization for LLMs.