Tool misuse, autonomous harm, and agent-specific attack vectors
Newest first · 1 reviewed on this page
Search insteadChuanchao Zang, Zi-Jian Cao, Xiangtao Meng +6
Long-term memory is becoming a core capability of LLM agents, enabling personalization and long-horizon interaction. However, memory mechanisms that retain, transform, or expose more information can affect both benign utility and susceptibility to memory poisoning. Existing…
Adil Khan, Khaled AlKhanbashi, Azza Mohamed
Large language model (LLM) agents that retrieve external content and use tools are vulnerable to indirect prompt injection, in which untrusted content contains instructions intended to influence agent behavior. We evaluated four defenses and an undefended control across GPT-5.4,…
Wu-Jie Xiong, Rabimba Karanjai, Yang Lu +2
Large language model agents place outputs from external skills into their execution context, allowing attacker-controlled data to influence later privileged actions. Existing defenses mainly classify untrusted content or authorize proposed operations. They do not directly…
Maurice E. Dawson, A. B. Ben Ayed, Samson Quaye
Industry 5.0 places autonomous agents inside its human–agent collaborative loop, resting its human-centricity pillar on an assumption of trustworthy collaboration that has not been examined critically. Agentic artificial intelligence has moved from research demonstration to…
Chenmin Wu, H. Jia, Yang Liu +11
Large language model agents are increasingly deployed as autonomous loops. Starting from one human goal, such a system repeatedly discovers work, plans, executes tool calls, verifies outcomes and persists state across many unattended iterations. The agent safeguards in wide use,…
CheolWon Na, Hao Ni, Lukasz Szpruch +7
LLM-based multi-agent trading systems, in which specialized agents collaborate through structured communication to produce trading decisions, are moving rapidly from research prototypes to live deployments that control real assets. The same inter-agent communication that makes…
Zhaowen Feng, Zhenhui Liu, Ming-Jun Ma +2
Tool-level attacks on Large Language Model (LLM) agents—poisoned tool descriptions, prompt injection, and capability misrepresentation—are universally effective, yet no existing defense provides comprehensive protection. We propose Architectural Intent Collapse (AIC), a formal…
Sana Mourad, E. E. Abdallah, Mohammad Ababneh
Current language model deployments face growing security challenges from prompt-based attacks, including jailbreaks, direct and indirect prompt injection, and instruction hijacking, which often evade traditional rule-based safeguards. As these models are increasingly integrated…
Anees Ahmed Mahaboob Ali, R. Delhibabu, Everette Jacob Remington Nelson
Background. Artificial-intelligence design tools now plan genome-scale edits, and agentic systems execute those plans with progressively less human oversight. Biosecurity controls are limited to two points: refusal guardrails at the foundation model and sequence-identity…
Gaurav Kukreja, Parul Kukreja, Mohammed Abraar +3
Large language models (LLMs) can generate plausible-sounding ETF portfolios while silently violating basic KYC-style constraints on risk, fees, and diversification. This is especially problematic in agentic multi-turn advisory systems, where each draft recommendation can become…
Xing Zhang, Ya Cui, Guanghui Wang +1
Long-form research reports generated by large language models drift, contradict themselves, and lose provenance: the same metric appears with different values, and rumor is quoted as confidently as an audited filing. We present a two-tier agentic system that separates a…
Quang-Vinh Dang, Hoang-Viet Vu, Ngoc-Son-An Nguyen +2
Evaluating the security posture of large language model (LLM) deployment stacks is a critical challenge in modern AI security. Traditional vulnerability management frameworks—such as the Common Vulnerability Scoring System (CVSS) and component-level checklists—assume that…
Ted Kwartler, Alan Aqrawi, Arian Abbasi
Long-running agents periodically compact their context, replacing the transcript with a model-generated summary. Recent work shows that dropping a standing safety constraint during compaction drives behavioral violations across many models (Governance Decay; Chen, 2026). We ask…
Gabriel Huang, Abhay Puri, L'eo Boisvert +4
Open-weight LLM agents are vulnerable to backdoors installed during fine-tuning, which may be undetectable if the trigger conditions are never met during testing. Assuming defenders do not know the existing trigger, they cannot unlearn it directly. One decontamination strategy…
Rohan Bhagra, Mahantesh Halapannavar, Uddhav Bhattarai
Advances in advanced artificial intelligence tools have sparked research in robot autonomy, but the development of such systems has largely focused on execution rather than verifying the feasibility actions planning models propose. Like general-purpose LLMs, robotics planning…
Hao Sui, Simeng Qin, Jie Liao +3
Agent skills, bundles of instructions and resources that an LLM agent loads on demand, form an emerging supply chain where a single poisoned skill can persistently compromise every agent that installs it. However, existing skill attacks either fire on every request or rely on…
Srinivas Telukunta, Georgios Nektarios Lilis, Lucio Baron
Enterprises are deploying autonomous AI agents faster than they can govern them, and prevailing approaches stretch a single discipline, typically DevSecOps built for deterministic automation, across every scale of agency. We argue that agentic AI governance is four problems, not…
Yuyang Luo, Haoran Wang, Kai Shu
Agentic skills improve large language model (LLM) agents by encoding reusable procedures for complex tasks. However, manually authored skills often adapt poorly to long-horizon tasks and changing environments. To address the limitation, self-evolving skill systems have been…
Sakshi Jain
For a decade, responsible AI at internet scale rested on a reassuring assumption: that risk lives primarily in a model, it is a unit you can isolate, and that privacy, fairness and safety is therefore something you certify at model level, before launch. At LinkedIn, where AI…
Qingqing Wang, Jiazhu Xie, Bowen Li +2
Small and medium-sized enterprises (SMEs) are increasingly deploying Large Language Model (LLM)-based agentic systems to support customer service and internal knowledge management. However, practical deployment of retrievalaugmented generation (RAG) systems continues to be…
Abdulkadir Külçe, A. Esen, Cağla Fikir +4
This paper presents ECHO (Enhanced Care&Health Observer), a locally-deployable conversational health assistant for long-term chronic care management. ECHO integrates three complementary software modules developed under shared supervision as a unified system. The core module is…
Ryozo Masukawa, Ian Bryant, Armita Kazeminajafabadi +6
Autonomous cyber defense systems based on Deep Reinforcement Learning (DRL) have attracted significant research attention, yet remain evaluated almost exclusively against static, heuristic red agents, leaving their robustness against adaptive threats critically understudied.…
Chenkai Zhang, Yiran Li, Yifang Tian +2
Reliability in LLM-based agentic systems is a property of the whole execution (its tool calls, model calls, guardrails, and inter-agent messages), not of the final answer alone, yet evaluating only task outcomes reveals little about how or why a run fails. We present…
C. Thapa, Mohan Baruwal Chhetri, M. Grobler +2
Frontier artificial intelligence (FAI), encompassing large-scale, general-purpose AI systems, including large language models, multimodal foundation models, and agentic systems, is increasingly integrated into critical infrastructure (CI). This challenges long-standing security…
Zheng Lin, Yuzhen Huang, Zhenxing Niu +2
Long-term memory enables LLM agents to retain useful information across sessions, but also creates an attack surface through which adversaries may poison an agent's persistent memory to steer its behavior. Existing memory poisoning attacks mainly rely on individually malicious…
Sushma Mahadevaswamy
The rapid enterprise adoption of agentic artificial intelligence (AI) has introduced a category of security risk that existing cyber security frameworks were not designed to address. With 78 per cent of Fortune 500 companies projected to deploy agentic AI by 2026 and the global…
Prudvi Saisaran Ponduru, Pavani Priya Vyshnavi Nandanavanam, S. Ponduru
Cloud infrastructure failures are increasingly difficult to detect, diagnose, and remediate because production environments combine microservices, Kubernetes control loops, service meshes, serverless workloads, infrastructure-as-code, continuous delivery, and heterogeneous…
David C. Flynn
Large language models (LLMs) have expanded far beyond traditional software boundaries, mediating decisions, generating authoritative sounding content, and directly influencing human reasoning across consumer, enterprise, and critical infrastructure domains. The scale and…
Nazar Waheed
Abstract Large language model (LLM) applications increasingly combine probabilistic language interpretation with retrieval, persistent memory, external tools, and delegated enterprise credentials. This creates a system-security problem in which untrusted semantic content can…
Saurabh Sharma
Abstract Prompt injection—the manipulation of an LLM-based agent through adversarial content embedded in user input—poses a critical security risk in production agentic systems with access to sensitive datastores and code execution environments. Detection-based defenses (keyword…
Aparnaa Mahalaxmi Arulljothi, Theepan Kumar Gandhi
With LLM agents increasingly deployed to autonomously processed external content like web pages, emails, documents, and API responses become targets of indirect prompt injection attacks where malicious instructions embedded in external content hijack the agent's behavior.…
Divya Pandey, Shivani Manchanda, Gangesh Pathak +1
Clinical large language model (LLM) agents are entering production hospital deployments, where they read longitudinal electronic health records (EHRs), retrieve evidence from clinical knowledge bases, and assist with summarization, dosing, triage, and guideline-based decisions.…
Wesley Gao
Tool-using language-model agents can convert indirect prompt injection into consequential actions, making guardrail quality a joint security, utility, and efficiency problem. This study evaluates a ReAct-style control, native tool filtering, deterministic self-verification, a…
Shaina Raza, Ranjan Sapkota, Manoj Karkee +1
Agentic AI systems, built upon large language models (LLMs) and deployed in multi-agent configurations, are redefining intelligence, autonomy, collaboration, and decision-making across enterprise and societal domains. This review presents a structured analysis of Trust, Risk,…
Domenico Ursino, Gianluca Bonifazi, Enrico Corradini +5
Agentic AI and Large Language Models (LLMs) are transforming how language is understood and generated while reshaping decision-making, automation, and research practices. LLMs provide underlying reasoning capabilities, and Agentic AI systems use them to perform tasks through…
Hengyu An, Jinghuai Zhang, Tianyu Du +4
Large language model (LLM) agents are widely deployed in real-world applications, where they leverage tools to retrieve and manipulate external data for complex tasks. However, when interacting with untrusted data sources (e.g., fetching information from public websites), tool…
Georgios Syros, Anshuman Suri, Cristina Nita-Rotaru +1
Large Language Model (LLM)-based agents increasingly interact, collaborate, and delegate tasks to one another autonomously with minimal human interaction. Industry guidelines for agentic system governance emphasize the need for users to maintain comprehensive control over their…
I. Habler, Ken Huang, Vineeth Sai Narajala +1
As Agentic AI systems evolve from basic workflows to complex multi-agent collaboration, robust protocols such as Google’s Agent2Agent (A2A) become essential enablers. To foster secure adoption and ensure the reliability of these complex interactions, understanding the secure…
Ken Huang, Vineeth Sai Narajala, J. Yeoh +5
Traditional Identity and Access Management (IAM) systems, primarily designed for human users or static machine identities via protocols such as OAuth, OpenID Connect (OIDC), and SAML, prove fundamentally inadequate for the dynamic, interdependent, and often ephemeral nature of…
Ibrahim Adabara, Bashir Olaniyi Sadiq, Aliyu Nuhu Shuaibu +2
Agentic Artificial Intelligence systems, characterized by autonomous reasoning, memory augmentation, and adaptive planning, are rapidly reshaping technological landscapes. Unlike traditional AI or large language models, agentic AI integrates decision-making with persistent…
Chen Henry Wu, Rishi Shah, Jing Yu Koh +3
Demonstrates adversarial attacks on multimodal agents that take actions in digital environments, showing visual perturbations can hijack agent behavior.
L'eo Boisvert, Mihir Bansal, Chandra Kiran Reddy Evuru +9
We present DoomArena, a security evaluation framework for AI agents. DoomArena is designed on three principles: 1) It is a plug-in framework and integrates easily into realistic agentic frameworks like BrowserGym (for web agents) and $\tau$-bench (for tool calling agents); 2) It…
Tobin South, Subramanya Nagabhushanaradhya, A. Dissanayaka +18
The rapid rise of AI agents presents urgent challenges in authentication, authorization, and identity management. Current agent-centric protocols (like MCP) highlight the demand for clarified best practices in authentication and authorization. Looking ahead, ambitions for highly…
Zhilong Wang, N. Nagaraja, Lan Zhang +3
LLM agents are widely used as agents for customer support, content generation, and code assistance. However, they are vulnerable to prompt injection attacks, where adversarial inputs manipulate the model’s behavior. Traditional defenses like input sanitization, guard models, and…
Jinyu Chen, Jixiao Yang, Ziyang Zeng +3
Large Language Model (LLM)-based multi-agent systems have demonstrated remarkable capabilities across diverse applications, yet they face critical security challenges including backdoor attacks, prompt injection, and privacy leakage. Existing defense mechanisms typically address…
S. Hossain, Ruksat Khan Shayoni, Mohd Ruhul Ameen +3
Prompt injection attacks represent a major vulnerability in Large Language Model (LLM) deployments, where malicious instructions embedded in user inputs can override system prompts and induce unintended behaviors. This paper presents a novel multi-agent defense framework that…
Chris Schmitz, Jonathan Rystrøm, Jan Batzner
This paper finds that the introduction of agentic AI systems intensifies existing challenges to traditional public sector oversight mechanisms -- which rely on siloed compliance units and episodic approvals rather than continuous, integrated supervision. We identify five…
Diego Gosmar, Deborah A. Dahl
This paper proposes a novel architectural framework aimed at enhancing security and reliability in multi-agent systems (MAS). A central component of this framework is a network of Sentinel Agents, functioning as a distributed security layer that integrates techniques such as…
Ali Eslami, Jiangbo Yu
Agentic AI is increasingly being explored and introduced in both manually driven and autonomous vehicles, leading to the notion of Agentic Vehicles (AgVs), with capabilities such as memory-based personalization, goal interpretation, strategic reasoning, and tool-mediated…
Edoardo Allegrini, Ananth Shreekumar, Z. B. Celik
Agentic AI systems, which leverage multiple autonomous agents and large language models (LLMs), are increasingly used to address complex, multi-step tasks. The safety, security, and functionality of these systems are critical, especially in high-stakes applications. However, the…