The authors modified DPO, which trains on prompt, preferred-response and rejected-response triples.
Token-aware DPO preserves useful behavior in rejected model responses
GAW-PO weakens penalties for rejected tokens whose gradient updates align with learning the preferred response.
Big Tech
Andreea Dutulescu · Stefan Ruseti · Mihai Masala · Traian Rebedea · Mihai Dascalu
National University of Science and Technology POLITEHNICA Bucharest · NVIDIA
Research Digest··2 min read
Dutulescu et al.
Why this paper
From NVIDIA and National University of Science and Technology POLITEHNICA Bucharest
In one line
GAW-PO reweights rejected tokens in DPO by gradient alignment with preferred updates, improving performance across 11 benchmarks.
What we could check
- ·No code link found
- ·No weights link found
- ·No dataset link found
- ·No compute details found
- ✓Limitations stated by the authors (2 noted)
- ·No benchmark numbers found
Observed from the paper text and links we have. Absence here means we did not find it, not that it does not exist.
§