Tag
Simple Preference Optimization
Posts and notes tagged with Simple Preference Optimization.
- July 21, 202614 min read
Simple Preference Optimization (SimPO)
How SimPO simplifies DPO: it replaces DPO's reward with the average log-likelihood that the model uses to generate text, which removes the reference model, and adds a target reward margin to separate preferred and rejected responses.