Tooling
Tool inventory and selection guide
15 tools across open-source, commercial, and AI-augmented BAS categories. Each review includes what the tool does well, what it doesn't do, and a "not for" section so you can rule it out fast.
open-source
-
garak
NVIDIA (community)
LLM vulnerability scanner. Runs probes spanning prompt injection, encoding bypasses, toxic generation, and known jailbreaks against a wide range of model endpoints, then reports per-probe pass/fail with examples.
automated-redteammodel-eval -
Inspect AI
UK AI Safety Institute
Framework for large-scale model evaluation with first-class support for agent and tool-use scenarios. Built by the UK AI Safety Institute; designed around the kinds of evaluations institutional safety teams actually need.
model-evalautomated-redteam -
LLM Guard
Protect AI
Input and output scanner library for LLM applications. Bundles PII detection, prompt-injection heuristics, toxicity scoring, secrets detection, code-execution detection, and similar checks that can be composed into pipelines.
guardrail-testing -
NeMo Guardrails
NVIDIA
Open-source toolkit for adding programmable rails (dialogue, safety, retrieval, execution) to LLM applications. Uses a Colang language for dialogue flows and integrates with model providers and retrieval layers.
guardrail-testing -
PyRIT
Microsoft
Python Risk Identification Toolkit for generative AI. Orchestrates attack prompts against target endpoints, supports multi-turn scoring, and integrates with custom converters and scorers for systematic adversarial evaluation.
automated-redteamprompt-testing -
Rebuff
Protect AI / community
Prompt-injection detection library. Combines heuristics, vector-similarity to a corpus of known injections, and an LLM classifier; used both as a runtime defense and as a probe to evaluate defenses.
guardrail-testingprompt-testing
commercial
-
Calypso AI
Calypso AI
AI security and governance platform with attention to enterprise policy enforcement and inspection of inputs/outputs across managed and SaaS AI tools. Heritage in U.S. public-sector AI assurance.
guardrail-testingmodel-eval -
Cymulate
Cymulate
Breach-and-attack simulation platform with broad coverage across email, endpoint, network, and cloud. Provides repeatable attack execution useful for validating detection engineering — including detections targeted at Shadow AI egress and AI-related abuse patterns.
BAS -
HiddenLayer
HiddenLayer
AI security platform with model scanning, monitoring, and adversarial testing. Has invested in supply-chain security for models and notable threat-intel content.
model-evalautomated-redteam -
Horizon3.ai NodeZero
Horizon3.ai
Autonomous penetration-testing platform that orchestrates real attacker tradecraft across enterprise IT infrastructure. AI-augmented decision making within the platform; valuable for the broader attack-surface context within which AI systems sit (an LLM agent's blast radius depends on the network and identity it can reach).
BASautomated-redteam -
Lakera Red
Lakera
Commercial automated red-teaming for LLM applications. Continuous, multi-turn adversarial testing with reporting suitable for security teams; pairs with Lakera Guard for runtime protection.
automated-redteam -
Mindgard
Mindgard
Automated AI red-teaming platform from a UK-academic-rooted vendor. Strong on attack libraries derived from current research and on integration into CI for continuous evaluation.
automated-redteammodel-eval -
Robust Intelligence
Cisco (Robust Intelligence)
Enterprise platform for AI risk: pre-deployment model validation, continuous monitoring, and runtime AI firewall. Covers both predictive ML and generative AI workloads with reporting designed for risk and compliance functions.
automated-redteammodel-evalguardrail-testing
freemium
-
Giskard
Giskard
Open-source testing framework for ML and LLM applications. Generates test suites that probe for performance, robustness, hallucination, bias, and prompt injection; supports a hub for collaboration and result tracking.
model-eval -
Promptfoo
Promptfoo
YAML-configured evaluation harness for LLM prompts and applications. Supports test cases, assertions, redteaming plugins, and side-by-side model comparison; integrates well with CI for regression evaluation and with both local and hosted model endpoints.
model-evalprompt-testingautomated-redteam