Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls

Abstract

Long-horizon tasks remain uncommon in large language model (LLM) evaluation, and for a reason: when each step depends on the last, per-step accuracy that looks excellent in isolation decays catastrophically, as errors cascade and the end-to-end failure probability grows sharply with length. Existing agentic benchmarks report end-to-end success but confound this state-tracking difficulty with instruction interpretation, give no control group that isolates it, and are vulnerable to shortcuts such

Categories

@misc{pai2026longhorizon,
  title = {{Long-Horizon State Tracking in LLMs: Executing MD5 through a Deep Sequence of Dependent Tool Calls}},
  author = {Dheeraj Mohandas Pai and Lu Xian},
  year = {2026},
  month = sep,
  eprint = {2609.00012},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2609.00012}
}