Security Document Classification with a Fine-Tuned Local Large Language Model: Benchmark Data and an Open-Source System

Abstract

Organizations that scan documents for sensitive information face a practical problem. Cloud services require data to be sent to external infrastructure, while rule-based tools often miss threats that depend on context. This study presents TorchSight, an open-source local system for security document classification built around a fine-tuned Qwen 3.5 27B model. The model was trained on 78,358 samples from 13 permissively licensed sources and GPT-4 synthetic data covering seven security categories

Categories

Framework mappings

Suggested from the entry's categories.

@misc{dobrovolskyi2026security,
  title = {{Security Document Classification with a Fine-Tuned Local Large Language Model: Benchmark Data and an Open-Source System}},
  author = {Ivan Dobrovolskyi},
  year = {2026},
  month = may,
  eprint = {2605.20368},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.20368}
}