Autonomous Operations
9 resourcesAgentic AI Security
Guardrails for autonomous agents, self-modification prevention, and containment
CyBiasBench: Benchmarking Bias in LLM Agents for Cyber-Attack Scenarios
Taein Lim, Seongyong Ju, Munhyeok Kim + 2 more
Large language models (LLMs) are increasingly deployed as autonomous agents in offensive cybersecurity. In this paper, we reveal an interesting phenomenon: different agents exhibit distinct attack patterns. Specifically, each agent exhibits an attack-selection bias, disproportionately concentrating its efforts on a narrow subset of attack families regardless of prompt variations. To systematically quantify this behavior, we introduce CyBiasBench, a comprehensive 630-session benchmark that evalua
LITMUS: Benchmarking Behavioral Jailbreaks of LLM Agents in Real OS Environments
Chiyu Zhang, Huiqin Yang, Bendong Jiang + 8 more
The rapid proliferation of LLM-based autonomous agents in real operating system environments introduces a new category of safety risk beyond content safety: behavior jailbreak, where an adversary induces an agent to execute dangerous OS-level operations with irreversible consequences. Existing benchmarks either evaluate safety at the semantic layer alone, missing physical-layer harms, or fail to isolate test cases, letting earlier runs contaminate later ones. We present LITMUS (LLM-agents In-OS
Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs
Guinan Su, Yanwu Yang, Xueyan Li + 1 more
The continued improvements in language model capability have unlocked their widespread use as drivers of autonomous agents, for example in coding or computer use applications. However, the core of these systems has not changed much since early instruction-tuned models like ChatGPT. Even advanced AI agents function on message exchange formats, successively exchanging messages with users, systems, with itself (i.e. chain-of-thought) and tools in a single stream of computation. This bottleneck to a
Security Risks in Tool-Enabled AI Agents: A Systematic Analysis of Privileged Execution Environments
Hardik Goel
Tool-enabled AI agents are increasingly deployed in cloud-hosted environments and offered as services, where they perform side-effecting operations through privileged tools within execution environments. While such agents enable powerful automation, the security implications of hosting autonomous agents in privileged execution environments are not yet fully explored. This paper presents a structured analysis of security risks associated with cloud-hosted AI agents. We introduce a taxonomy of ris
Toward Web 4.0: Bidirectional Trust between AI Agents and Blockchain
Yunfeng Xia, Chao Li, Lei Li + 4 more
Autonomous AI agents are increasingly deployed on blockchain platforms, yet the design space that governs their interaction remains poorly understood. This convergence, where autonomous agents operate on and within decentralized systems, is a defining feature of the emerging Web~4.0 paradigm. This paper presents a Systematization of Knowledge organized around a bidirectional trust framework. In the B $\boldsymbol{\rightarrow}$ A direction, we examine how blockchain provides trust infrastructure
LLM Agents Can Autonomously Hack Websites
Richard Fang, Rohan Bindu, Akul Gupta + 2 more — arXiv preprint
Demonstrates that LLM agents can autonomously perform web hacking tasks including SQL injection, XSS, and CSRF attacks without human guidance.
SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering
John Yang, Carlos E. Jimenez, Alexander Wettig + 4 more — NeurIPS 2024
Demonstrates autonomous coding agents that interact with computer interfaces to solve software engineering tasks, raising questions about agent containment.
LLM Agents Can Autonomously Exploit One-day Vulnerabilities
Richard Fang, Rohan Bindu, Akul Gupta + 1 more — arXiv preprint
Shows that LLM agents (GPT-4) can autonomously exploit real-world one-day vulnerabilities given CVE descriptions, achieving 87% success rate.
Voyager: An Open-Ended Embodied Agent with Large Language Models
Guanzhi Wang, Yuqi Xie, Yunfan Jiang + 5 more — NeurIPS 2023
Demonstrates a continuously learning LLM agent in Minecraft that writes and executes code, highlighting autonomous operation and containment challenges.