DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning

Abstract

Task-specific fine-tuning can improve the performance of large language models (LLMs) on downstream tasks. However, our study reveals that task-specific fine-tuning can also weaken the safety guardrails of aligned LLMs. A widely adopted strategy for preserving safety during fine-tuning is to incorporate safety data. Although previous studies have shown that randomly mixing safety data can alleviate safety degradation, the underlying principle determining why some safety examples are more effecti

Categories

@misc{zhang2026datarx,
  title = {{DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning}},
  author = {Junbo Zhang and Qian-Li Zhou and Xin-Yang Deng and Wen Jiang},
  year = {2026},
  month = aug,
  eprint = {2608.04322},
  archivePrefix = {arXiv},
  url = {https://www.semanticscholar.org/paper/821fc65cadf063781de7fa906391711eba1c8374}
}