Tool misuse, autonomous harm, and agent-specific attack vectors
Newest first · 7 reviewed on this page
Search insteadYedidel Louck, Ariel Stulman, Amit Dvir
Multi-agent systems (MAS) powered by artificial intelligence (AI) are increasingly foundational to complex, distributed workflows. Yet, the security of their underlying communication protocols remains critically under-examined. This paper presents the first empirical,…
Hammad Atta, M. Baig, Yasir Mehmood +5
We introduce Cognitive Degradation as a novel vulnerability class in agentic AI systems. Unlike traditional adversarial external threats such as prompt injection, these failures originate internally, arising from memory starvation, planner recursion, context flooding, and output…
Sotiris Chatzimiltis, Mahdi Boloursaz Mashhadi, Mohammad Shojafar +2
Agentic AI systems are emerging as powerful tools for automating complex, multi-step tasks across various industries. One such industry is telecommunications, where the growing complexity of next-generation radio access networks (RANs) opens up numerous opportunities for…
P. Zambare, Venkata Nikhil Thanikella, Ying Liu
When combining Large Language Models (LLMs) with autonomous agents, used in network monitoring and decision-making systems, this will create serious security issues. In this research, the MAESTRO framework consisting of the seven layers threat modeling architecture in the system…
N. Petrovic, D. Krstić, S. Suljović +2
Internet of Things (IoT) devices are becoming an important part of our environment - from home applications to smart city infrastructures. Therefore, secure deployment and operation of IoT-based applications is one of the crucial factors for practical adoption of such services.…
Zimo Ji, Xunguang Wang, Zongjie Li +6
Large Language Model (LLM)-based agents with function-calling capabilities are increasingly deployed, but remain vulnerable to Indirect Prompt Injection (IPI) attacks that hijack their tool calls. In response, numerous IPI-centric defense frameworks have emerged. However, these…
Eranga Bandara, Amin Hass, Ross Gore +8
AI agent-based systems are becoming increasingly integral to modern software architectures, enabling autonomous decision-making, dynamic task execution, and multimodal interactions through large language models (LLMs). However, these systems introduce novel and evolving security…
Paschal C. Amusuo, Dongge Liu, Ricardo Andrés Calvo Méndez +3
Fuzz testing has become a cornerstone technique for identifying software bugs and security vulnerabilities, with broad adoption in both industry and open-source communities. Directly fuzzing a function requires fuzz drivers, which translate random fuzzer inputs into valid…
Kai Zhou, Ahmed Elgohary, S. M. Iftekhar +80
The ability of LLM agents to plan and invoke tools exposes them to new safety risks, making a comprehensive red-teaming system crucial for discovering vulnerabilities and ensuring their safe deployment. We present SIRAJ: a generic red-teaming framework for arbitrary black-box…
Toqeer Ali Syed, M. R. Belgaum, Salman Jan +2
Software supply chain attacks increasingly target trusted development and delivery processes, making conventional post-build integrity mechanisms insufficient. Existing frameworks like SLSA, SBOM, and in-toto mainly provide provenance and traceability but cannot actively…
Avihay Cohen
Large Language Model (LLM) based agents integrated into web browsers (often called agentic AI browsers) offer powerful automation of web tasks. However, they are vulnerable to indirect prompt injection attacks, where malicious instructions hidden in a webpage deceive the agent…
Prakhar Sharma, Haohuang Wen, V. Yegneswaran +3
The evolution toward 6G networks is being accelerated by the Open Radio Access Network (O-RAN) paradigm—an open, interoperable architecture that enables intelligent, modular applications across public telecom and private enterprise domains. While this openness creates…
Vaishali Vinay
Cybersecurity operations are increasingly adopting agentic AI solutions due to the time-critical and complex decision-making in security operations centers (SOCs). While large language models (LLMs) are good with summarization tasks or interpreting structured and unstructured…
Sunil Kumar Jang Bahadur, Gopala Dhar
Generative Artificial Intelligence (GenAI) presents significant advancements but also introduces novel security challenges, particularly within agentic workflows where AI agents operate autonomously. These risks escalate in multi-agent systems due to increased interaction…
A. Rodríguez, Jaime C. Acosta, Anantaa Kotal +1
Designing realistic and adaptive networked threat scenarios remains a core challenge in cybersecurity research and training, still requiring substantial manual effort. While large language models (LLMs) show promise for automated synthesis, unconstrained generation often yields…
V. Nguyen, M. Husain
Agentic AI introduces security vulnerabilities that traditional LLM safeguards fail to address. Although recent work by Unit 42 at Palo Alto Networks demonstrated that ChatGPT-4o successfully executes attacks as an agent that it refuses in chat mode, there is no comparative…
Liming Lu, Xiang Gu, Junyu Huang +5
Large Language Models (LLMs) are increasingly used in agentic systems, where their interactions with diverse tools and environments create complex, multi-stage safety challenges. However, existing benchmarks mostly rely on static, single-turn assessments that miss…
Anshuman Chhabra, Shrestha Datta, Shahriar Kabir Nahin +1
Agentic AI systems powered by large language models (LLMs) and endowed with planning, tool use, memory, and autonomy, are emerging as powerful, flexible platforms for automation. Their ability to autonomously execute tasks across web, software, and physical environments creates…
Kaleb Bacztub, Dylan Christensen, Arun Ravindran +1
This paper presents AutoPwn, an AI-enabled framework for automatic code-reuse exploit generation. AutoPwn leverages agentic large language models to orchestrate the classical stages of exploitation—gadget discovery, semantic analysis, chain construction, and payload…
Hanrong Zhang, Jingyuan Huang, K. Mei +5
Although LLM-based agents, powered by Large Language Models (LLMs), can use external tools and memory mechanisms to solve complex real-world tasks, they may also introduce critical security vulnerabilities. However, the existing literature does not comprehensively evaluate…
Richard Fang, Rohan Bindu, Akul Gupta +2
Demonstrates that LLM agents can autonomously perform web hacking tasks including SQL injection, XSS, and CSRF attacks without human guidance.
Richard Fang, Rohan Bindu, Akul Gupta +1
Shows that LLM agents (GPT-4) can autonomously exploit real-world one-day vulnerabilities given CVE descriptions, achieving 87% success rate.
Xiangming Gu, Xiaosen Zheng, Tianyu Pang +5
A multimodal large language model (MLLM) agent can receive instructions, capture images, retrieve histories from memory, and decide which tools to use. Nonetheless, red-teaming efforts have revealed that adversarial images/prompts can jailbreak an MLLM and cause unaligned…
Edoardo Debenedetti, Jie Zhang, Mislav Balunovic +3
Introduces AgentDojo, a framework for evaluating the security of LLM agents against prompt injection and other attacks in realistic tool-use scenarios.
Yujun Zhou, Yufei Han, Haomin Zhuang +5
Large Language Models (LLMs) demonstrate remarkable capabilities across diverse applications. However, concerns regarding their security, particularly the vulnerability to jailbreak attacks, persist. Drawing inspiration from adversarial training in deep learning and LLM agent…
Tongxin Yuan, Zhiwei He, Lingzhong Dong +9
Introduces R-Judge benchmark for evaluating whether LLM agents can identify safety risks in agentic scenarios involving tool use and multi-step reasoning.
Feng He, Tianqing Zhu, Dayong Ye +3
Surveys security and privacy challenges specific to LLM-based agents, covering agent architectures, attack surfaces, and defense mechanisms.
Ayush RoyChowdhury, Mulong Luo, Prateek Sahu +2
Introduces confused deputy attacks against RAG-based code assistants like GitHub Copilot, where poisoned code repositories manipulate assistant outputs.
Kai Greshake, Sahar Abdelnabi, Shailesh Mishra +3
Introduces indirect prompt injection attacks against LLM-integrated applications, demonstrating how adversaries can remotely control LLMs by injecting prompts into data sources the LLM retrieves.
Karthik Perikala
Enterprise adoption of generative AI is rapidly shifting from isolated prompt-driven applications toward complex agentic systems that integrate retrieval, reasoning, and tool execution. As these systems grow in scale, the lack of a standardized interaction model between agents…