Beyond the Black Box: Interpretability of Agentic AI Tool Use

Hariom Tatsat, Ariye Shater

Abstract

AI agents are promising for high-stakes enterprise workflows, but dependable deployment remains limited because tool-use failures are difficult to diagnose and control. Agents may skip required tool calls, invoke tools unnecessarily, or take actions whose consequence becomes visible only after execution. Existing observability methods are mostly external: prompts reveal correlations, evaluations score outputs, and logs arrive only after the model has already acted. In long-horizon settings, thes

Categories

Cite This Resource

@article{llmsec202600178,
  title = {Beyond the Black Box: Interpretability of Agentic AI Tool Use},
  author = {Hariom Tatsat and Ariye Shater},
  year = {2026},
  url = {https://arxiv.org/abs/2605.06890},
}