Zelong Xu

2026

1. DOG-DPO: Dynamic Optimization in Geometry for Safety Alignment

   Yi Nian, Tiankai Yang, Yudi Zhang, Qi Pan, Zelong Xu, Shenzhe Zhu, Qingqing Luan, Yue Huang, Xiangliang Zhang, Yue Zhao

   Findings of EMNLP 2026 arXiv:2606.07678

   arXivPDF

   Abstract

   Safety alignment for large language models relies on preference data, but current pipelines often train on large, redundant datasets. Existing data selection methods typically score each preference pair independently, collapsing directional preference information into scalar quality or diversity scores. This sample-centric view is especially limiting in multi-dataset settings, where shared safety directions coexist with dataset-specific residual risks. We propose DOG-DPO, a training-free data selection framework that treats preference pairs as structured geometric signals. DOG-DPO first represents each preference pair as a direction in model representation space. It then decomposes multi-dataset preference geometry into a global anchor subspace and dataset-specific residual subspaces. Finally, it selects subsets by maximizing diversity-based coverage, encouraging broad, non-redundant coverage of alignment directions before DPO training. Across six safety benchmarks and two model backbones, DOG-DPO achieves a strong utility-robustness trade-off using only 11% of the preference pairs. It recovers most of the safety gains of full-data training while remaining entirely teacher-free, training-free, and substantially faster than representative selection baselines.

   BibTeX

   @inproceedings{nian2026dog,
     title        = {DOG-DPO: Dynamic Optimization in Geometry for Safety Alignment},
     author       = {Yi Nian and Tiankai Yang and Yudi Zhang and Qi Pan and Zelong Xu and Shenzhe Zhu and Qingqing Luan and Yue Huang and Xiangliang Zhang and Yue Zhao},
     booktitle    = {Findings of EMNLP 2026},
     year         = {2026},
     url          = {https://arxiv.org/abs/2606.07678},
   }

   Copy