Anomaly detection, behavioral analysis, and drift monitoring
Newest first · 1 reviewed on this page
Search insteadLaksh Advani
Autonomous LLM agents generate multi-step action plans that can fail due to contextual misalignment or structural incoherence. Existing anomaly detection methods are ill-suited for this challenge: mean-pooling embeddings dilutes anomalous steps, while contrastive-only approaches…
Zhijie Zheng, Yu Li, Chen Qian +5
LLM-based agents can interact with external environments through tool invocation, but this capability also introduces security risks such as file modification, information leakage, and unauthorized actions. Existing guardrails often evaluate completed trajectories, leaving…
Maikel Leon
Generative AI adoption has outpaced organizational governance capabilities. We conceptualize AI guardrails as sociotechnical governance mechanisms, comprising policy, technical, and workflow components that embed organizational norms in deployed AI systems. Extending norm-based…
Venkata Sai Abhinav Piratla -
The integration of artificial intelligence into life-critical medical device controllers—including closed-loop insulin delivery systems and cardiac monitoring devices—introduces adversarial machine learning (AML) attack surfaces that conventional cybersecurity frameworks do not…
Sultan Zavrak
The Model Context Protocol (MCP) has become a widely adopted interface for LLM agents to invoke external tools, yet learned monitoring of MCP tool-call traffic remains underexplored. In this article, MCPShield is presented as an attack detection framework for MCP tool-call…
Yizhe Zeng, Wei Zhang, Yunpeng Li +3
While Chain-of-Thought (CoT) prompting has become a standard paradigm for eliciting complex reasoning capabilities in Large Language Models, it inadvertently exposes a new attack surface for backdoor attacks. Existing CoT backdoor attacks typically manipulate the intermediate…
Jingshen Zhang, Bo Wang, Yanlin Fu +4
In this paper, we study an emergent self-debiasing mechanisms against stereotypical content in Large Language Models (LLMs). Unlike traditional safety mechanisms that are primarily triggered by explicit input-level stimuli, self-debiasing mechanisms can involve generation-time…
Kaung Myat Naing, Talha Ali, Mohammed Ouannass
Organizations outside formal governance frameworks often lack cybersecurity audit tools, making anomaly detection and risk evaluation difficult. This paper presents an AI-enhanced auditing framework for non-governance IT environments. Using the UNSW-NB15 dataset, we evaluate…
He Yang Yuan, Xin Wang, Kundi Yao +3
Logging code plays an important role in software systems by recording key events and behaviors, which are essential for debugging and monitoring. However, insecure logging practices can inadvertently expose sensitive information or enable attacks such as log injection, posing…
Xuanli He, Bilgehan Sel, Faizan Ali +3
Large Language Models (LLMs) are increasingly exposed to adaptive jailbreaking, particularly in high-stakes Chemical, Biological, Radiological, and Nuclear (CBRN) domains. Although streaming probes enable real-time monitoring, they still make systematic errors. We identify a…
Farbod Zorriassatine, Ahmad Lotfi
Agentic AI, with goal-directed, proactive, and autonomous decision-making capabilities, offers a compelling opportunity to address movement-related risks in human activity, including the persistent hazard of falls among elderly populations. Despite numerous approaches to fall…
Xiaomeng Hu, Yinger Zhang, Fei Huang +7
AI agents are expected to perform professional work across hundreds of occupational domains (from emergency department triage to nuclear reactor safety monitoring to customs import processing), yet existing benchmarks can only evaluate agents in the few domains where public…
C. V. Suresh Babu, S. Nanda Kumar, S. Abhishek +1
This chapter per the authors aims to advance transparency and trust in data and AI ecosystems by examining explainability, evaluation, and continuous monitoring for classical machine learning models and large language models. The target groups include researchers, AI…
Zedian Shao, Charles Fleming, Teodora Baluta
Large language models (LLMs) are often fine-tuned on uncurated text datasets that adversaries can poison. Existing poisoning attacks primarily rely on fixed trigger phrases that defenses such as outlier detection, clean-data regularization, or online monitoring can neutralize.…
Kavana Venkatesh, Jafar Isbarov, Saad Amin +2
Cascade attacks in LLM multi-agent systems (MAS) arise when adversarial influence propagates across agents and leads to escalated system-level failures through complex agent interactions. Detecting such cascades is challenging, as their signals are distributed, tightly coupled…
Shuze Liu, Qianwen Guo, Yushun Dong
Large language models (LLMs) are increasingly deployed through hosted APIs, making model extraction a practical threat to model ownership and service security. However, individual extraction queries often resemble benign requests, and existing evaluations often focus on…
Davis Brown, Samarth Bhargav, Arav Santhanam +7
Language models can find thousands of severe software vulnerabilities, and agents are increasingly being misused for cyberattacks. To avoid detection, attackers frequently distribute their misuse, splitting a harmful task across many user accounts so each individual transcript…
Thanh Luong Tuan, Abhijit Sanyal
Pre-deployment verification of enterprise artificial intelligence (AI) agents remains a critical gap between large language model (LLM) capability benchmarking and production deployment. Post-deployment monitoring, human-in-the-loop controls, and prompt-level guardrails offer…
Alaia Solko-Breslin, Pramod Kaushik Mudrakarta, Mihai Christodorescu +2
Securing AI agents that operate in complex digital environments has become a critical need, and runtime monitoring approaches that formulate and enforce policies expressed in a formal language like Datalog offer a promising solution. However, existing approaches are restricted…
Jennifer Za, Julija Bainiaksina, Nikita Ostrovsky +2
Chain-of-thought (CoT) monitoring is a promising safety mechanism for AI agents, based on the premise that visible reasoning traces can surface misaligned or deceptive behavior. While effective in standard scenarios, recent work highlights that LLMs remain vulnerable to…
Saroj Gopali, Bipin Chhetri, Deepika Giri +2
Cyberattacks on operational technology are increasingly causing costly downtime and physical damage, exposing the limitations of traditional rule-based monitoring in industrial IoT environments. While Large Language Models (LLMs) have strong semantic reasoning abilities to…
Preeti Ravindra, Rahul Tiwari, Vincent Wolowski
AI software development agents are increasingly capable of modifying infrastructure and security critical systems, creating risks where an agent completes its assigned task while covertly weakening safeguards through actions such as broadening permissions, degrading logging, or…
Yousef Emami, Rahim Taheri, Mohammadhossein Homaei +2
Uncrewed Aerial Vehicle (UAV) swarms have significant potential for applications such as Search and Rescue (SAR) and environmental monitoring, but their real-world deployment is limited by a lack of situational awareness, intermittent connectivity, and significant cybersecurity…
Jie Li
Large language model (LLM) agents are starting to take on routine work in high-performance computing (HPC), including monitoring Slurm jobs, diagnosing failed builds, inspecting simulation output, and coordinating scientific workflows. To do this work, an agent commonly acts…
Lena Libon, Ben Rank, Jehyeok Yeon +5
As AI agents begin to automate AI R&D, we need ways to assess whether their outputs are safe to deploy, even when the agents themselves may be untrusted. AI control offers one such approach: rather than trusting the agent, it treats it as a potential adversary and uses a monitor…
Pallavi Singh, Khushboo Gupta, Pratibha Singh
Large language model (LLM) agents extend generative models with planning, memory, and external tool access, but this capability creates a security path in which untrusted content can alter instructions, hijack an agent's operational goal, and trigger harmful tool actions. This…
Christopher Henshaw, Gour Karmakar
Existing approaches to anomalous behaviour log detection, such as Wazuh rely primarily on predefined detection rules, while statistical anomaly detection approaches such as OpenSearch identify deviations from previously observed behavioural patterns. Recent research has…
Libin Liu, Wenzhou Yang, Li Chen +2
Border Gateway Protocol (BGP) security is critical to Internet infrastructure, yet progress in routing anomaly detection has been limited by the scarcity of publicly available incident datasets, which contain only 18 recorded cases. We observe that public operator mailing lists,…
Timothy Merritt, Alejandro Jarabo-Peñas, Juan Bravo-Arrabal +2
Multi-drone systems are increasingly positioned for safety-critical missions such as search and rescue (SAR) and critical infrastructure monitoring. Yet, real-world adoption remains constrained not only by autonomy performance, but by the difficulty of integrating agentic…
Chaoyu Zhang, Hexuan Yu, Heng Jin +6
Agentic AI systems execute complex tasks through long-horizon workflows of planning, tool use, and multi-agent coordination. Task failures in these systems often originate from a single step, such as an injected prompt or a flawed plan, and are then amplified through downstream…
Rui Wen, Ahmed Salem, Andrew Paverd +2
Large language models are often fine-tuned, shared, or downloaded from third parties, so a deployed model may carry a hidden backdoor that behaves normally on benign inputs but switches to attacker-controlled behavior when a secret trigger appears. While backdoors can be audited…
Yuqiao Tan, Shizhu He, Jun Zhao +1
While research on recursive self-improvement (RSI) has predominantly automated model training pipelines, reliable autonomous development demands a missing pillar: post-hoc monitoring and auditing to understand what models learn and ensure safe alignment. Mechanistic…
Hanling Wang, Chenlong Wei, Ling Xu +6
As large language models (LLMs) are increasingly deployed, the generation of harmful content has become a critical safety concern. Existing safeguards operate at the input, output, or streaming-generation stages, while early-risk methods that rely on surface tokens or output…
Mufti Mahmud, Noushath Shaffi, M. S. Kaiser +12
Healthcare environments present uniquely demanding constraints for the deployment of artificial intelligence (AI). Clinical decisions of significant consequence are rarely the product of a single isolated computation; rather, safe and effective patient care relies on the…
J. Ponsam, Nalam Siva Bhadra, Jai Kushal Bysani
The wide use of the artificial intelligence (AI) in serious applications has led to the problem of the safe management of the cycles models of the AI, such as training, running, and monitoring. The management of the models is usually faced with the problem of the trade-off…
Hongbo Cao, Yongqi Sun, Li Duan +2
With the advancement of artificial intelligence, vision-language models (VLMs) that integrate text and image modalities have become central to multimodal learning and are increasingly deployed in Internet of Things (IoT) environments such as smart surveillance, autonomous…
Prudvi Saisaran Ponduru, Pavani Priya Vyshnavi Nandanavanam, S. Ponduru
Cloud infrastructure failures are increasingly difficult to detect, diagnose, and remediate because production environments combine microservices, Kubernetes control loops, service meshes, serverless workloads, infrastructure-as-code, continuous delivery, and heterogeneous…
C. Satya Kumar, Asha Sunki, Vinith Koppera +1
Developments in communication technology have facilitated more data sharing in geographically dispersed settings, but they have also enlarged the attack surface, raising questions about network security. Research focuses on AI-based anomaly detection systems to improve Network…
Diego Gosmar, Deborah A. Dahl
This paper proposes a novel architectural framework aimed at enhancing security and reliability in multi-agent systems (MAS). A central component of this framework is a network of Sentinel Agents, functioning as a distributed security layer that integrates techniques such as…
P. Zambare, Venkata Nikhil Thanikella, Ying Liu
When combining Large Language Models (LLMs) with autonomous agents, used in network monitoring and decision-making systems, this will create serious security issues. In this research, the MAESTRO framework consisting of the seven layers threat modeling architecture in the system…
DeadBits
Open-source scanner for detecting prompt injections using vector similarity, YARA rules, text classifiers, and canary tokens.