A Three-Axis Stress Test of LLM vs Classical ML for Network Intrusion Detection under Distribution Shift and Adversarial Evasion

Abstract

Large language models are increasingly benchmarked against classical machine learning for network intrusion detection (NIDS), almost always using same-dataset evaluation, and that protocol turns out to be incomplete. Evaluating XGBoost and RoBERTa-LoRA on two independently collected NetFlow v2 networks across three axes (same-dataset performance, cross-dataset transfer, and adversarial evasion) reveals no universal winner. The two models are statistically tied same-dataset. XGBoost wins decisive

Categories

@misc{atif2026threeaxis,
  title = {{A Three-Axis Stress Test of LLM vs Classical ML for Network Intrusion Detection under Distribution Shift and Adversarial Evasion}},
  author = {Muhammad Ebad Atif and Muhammad Haider Ali},
  year = {2026},
  month = sep,
  eprint = {2609.13511},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.13511}
}