In August 2026, AI red teaming shifted from "nice to have" to "can't ship without it." The EU AI Act's adversarial robustness testing mandate for high-risk systems became enforceable this month, OWASP released ASI 2026 (security testing standards for agentic applications), and this week OpenAI paused its Astra autonomous test over loss-of-control risks—red teaming suddenly became mandatory for every AI team. But when you actually start, the first question is which tool: promptfoo leads with 24k stars, DeepEval feels like pytest, NVIDIA's Garak scans for vulnerabilities, Microsoft's PyRIT and the UK government's Inspect AI bring official backing, and NeMo Guardrails sounds like red teaming but is really a runtime guardrail. This review lays all six side by side.
Scope first: this is based on each tool's GitHub README, official site, and public reviews from Kosmoy/Confident AI, as of 2026-08-10. Star counts verified via GitHub API (promptfoo 24,090★, DeepEval 17,493★, Garak 8,746★, NeMo Guardrails 6,900★, PyRIT 4,270★, Inspect AI 2,514★). This is a representative comparison, not a personal benchmark; pricing per each official site. This site also published the Astra pause hotspot (why red teaming suddenly matters) and the red teaming SOP (how to use the tool you pick)—this piece covers the "selection" step.
One: Why AI Red Teaming in 2026
Three forces pushed red teaming into a must-have. First, the EU AI Act's compliance requirements for high-risk AI systems (healthcare, hiring, credit scoring, etc.) entered the enforceable phase in August 2026, explicitly requiring adversarial robustness testing—not a "recommendation," but fail-and-it's-illegal. Second, OWASP released ASI 2026 (Agentic Security Initiative), shifting the testing focus from single-turn prompts to agentic applications: agents can call tools and execute multi-step flows, so the attack surface is an order of magnitude wider than a chatbot, and traditional "input filtering" no longer suffices. Third, this week OpenAI paused its Astra autonomous test over loss-of-control risks (see this site's hotspot analysis)—a top lab's internal red team couldn't catch it in advance, showing current test coverage is far from enough.
The result: red teaming went from "an optional hardening step before launch" to "a mandatory gate before launch." But which tool to use—there's no industry consensus yet. That's what this piece breaks down.
Two: Six-Tool Comparison—Specs Table
Six representative tools, compared across positioning, stars, language, license, vendor, and best scenario.
| Tool | Positioning | Stars | Language | License | Vendor | Best for |
|---|---|---|---|---|---|---|
| promptfoo | Prompt/agent/RAG testing + red team | 24,090★ | TypeScript | MIT | Community | CI-integrated red team + eval |
| DeepEval | LLM evaluation framework | 17,493★ | Python | Apache-2.0 | Confident AI | pytest-style evaluation |
| Garak | LLM vulnerability scanner | 8,746★ | Python | Apache-2.0 | NVIDIA | Vulnerability probing/jailbreak scan |
| NeMo Guardrails | Runtime guardrails | 6,900★ | Python | Apache-2.0 | NVIDIA | Input/output protection (not red team) |
| PyRIT | Generative AI risk identification | 4,270★ | Python | MIT | Microsoft | Automated multi-turn adversarial probing |
| Inspect AI | LLM evaluation framework | 2,514★ | Python | MIT | UK Gov AISI | Rigorous research-grade eval |
A few clarifications. First, promptfoo has the most stars (24k) but it's TypeScript; the other five are Python—check your stack before selecting. Second, vendor backgrounds fall into four buckets: community (promptfoo), startup (DeepEval, backed by Confident AI), big tech (NVIDIA's two + Microsoft's PyRIT), and government (the UK AISI's Inspect AI). Third, a key insight: not all six are red teaming tools. promptfoo/PyRIT/Garak lean toward offensive red teaming, DeepEval/Inspect AI lean toward evaluation, and NeMo Guardrails is a runtime guardrail—three different things, broken down below.
Three: One by One—Each Tool's Best Range
promptfoo: simplest config, top pick for CI red teaming. 24,090★, TypeScript, MIT. Positioned as "test your prompts, agents, and RAGs, with red teaming/pentesting"—a single YAML config runs everything: declare the target, red team strategies (jailbreak, prompt injection, PII leakage, etc.), and test count; one CLI command generates attack cases and scores them. Its strength is CI-friendliness: runs in GitHub Actions, outputs JSON/HTML reports, and can gate releases on a pass threshold. Shortcoming: evaluation metrics aren't as granular as DeepEval—it leans toward "was it broken" rather than "how bad was the output after being broken." Best for: engineering teams wanting red teaming inside their CI pipeline. Minimal config example:
# promptfoo red team minimal config
description: "Customer support bot red team test"
targets:
- file://my-chatbot.py
redteam:
purpose: "Banking customer support chatbot"
strategies:
- prompt-extraction
- jailbreak
- pii-leakage
numTests: 20DeepEval: pytest-style evaluation, most comprehensive metrics. 17,493★, Python, Apache-2.0, backed by Confident AI. An LLM evaluation framework offering 50+ metrics (faithfulness, relevance, toxicity, bias, etc.), written just like pytest—decorate test cases, assert on metric scores. Its strength is granular evaluation: not just "did it answer right," but "did the answer hallucinate, is it biased." Shortcoming: red teaming is an add-on (it has a red teaming module but less specialized than PyRIT/Garak), leaning more toward "quality evaluation" than "attack simulation." Best for: teams that want to quantify model quality and write evaluation test suites.
Garak: LLM vulnerability scanner, broadest probe coverage. 8,746★, Python, Apache-2.0, from NVIDIA. Clearly positioned as "the LLM vulnerability scanner," with a large library of probing modules: jailbreaks, data leakage, prompt injection, malicious code generation, and more—one command runs the full probe suite against a model and reports which attack surfaces were broken. Its strength is broad attack-surface coverage, out of the box. Shortcoming: it's more "scanning" than "evaluation"—it tells you where the leaks are, not how good the answers are; config flexibility trails promptfoo. Best for: a one-time vulnerability checkup before launch.
NeMo Guardrails: runtime guardrail, not a red teaming tool. 6,900★, Python, Apache-2.0, from NVIDIA. Positioned as "add guardrails to LLM applications"—it intercepts between input and output: filters malicious prompts, restricts topics, controls dialogue flow. Key insight: it's a protection layer, not a testing tool. Red teaming "finds vulnerabilities before launch"; guardrails "block attacks after launch." It's included in this comparison to draw the boundary—don't assume that installing Guardrails means you've done red teaming. Best for: real-time protection for live applications, used alongside red teaming tools, not as a replacement.
PyRIT: Microsoft's red teaming tool, strongest multi-turn adversarial. 4,270★, Python, MIT, from Microsoft. Full name Python Risk Identification Tool for generative AI; it automates multi-turn adversarial probing—not a single-turn prompt throw, but simulating multi-turn dialogue that escalates the attack step by step. Its strength is multi-turn adversarial: closer to real attacker behavior, well-suited for testing agentic applications. Shortcoming: research-script style, docs less friendly than promptfoo, CI integration requires custom wrapping. Best for: research-oriented teams, scenarios testing multi-turn agent security.
Inspect AI: from the UK government, most rigorous research-grade eval. 2,514★, Python, MIT, from the UK government's AISI (AI Safety Institute). An LLM evaluation framework—the same family of tools used by the people testing OpenAI's Astra (the safety testing behind this week's Astra pause hotspot relies on this type of framework). Its strength is methodological rigor: supports complex evaluation workflows, dataset management, and reproducibility. Shortcoming: lowest star count, smallest community, setup leans toward a research framework rather than an engineering tool. Best for: academic/policy research, institutions needing rigorous reproducible evaluation.
Four: Red Team / Evaluation / Guardrails—Don't Confuse the Three
This is the most important insight in this piece: these three categories solve different problems.
- Red teaming (probing/attack): before launch, actively simulate attacks to find vulnerabilities. Representatives: promptfoo, PyRIT, Garak. Asks "can it be broken."
- Evaluation (eval/metrics): measure whether model quality and behavior meet expectations. Representatives: DeepEval, Inspect AI. Asks "how well does it answer."
- Guardrails (runtime protection): after launch, intercept malicious inputs and outputs in real time. Representative: NeMo Guardrails. Asks "how to block."
The most common mistake: installing NeMo Guardrails and assuming "we've done red teaming"—guardrails block known patterns, red teaming finds unknown vulnerabilities, and the two are not interchangeable. A complete security practice is: red team to find vulnerabilities → evaluate to quantify impact → guardrails to block attacks → red team again to test whether the guardrails can be bypassed. This site's red teaming SOP covers how to operationalize this flow.
Five: Selection Advice—Pick by Need
Second table, by need, with direct conclusions.
| Your need | Top pick | Reason |
|---|---|---|
| Red teaming in CI, simplest config | promptfoo | YAML config, CI-friendly, top stars |
| Quantify model quality, write eval suites | DeepEval | 50+ metrics, pytest-style, most comprehensive |
| One-time vulnerability checkup before launch | Garak | Vulnerability scanner, broad coverage, out of the box |
| Test multi-turn agent security | PyRIT | Microsoft-backed, multi-turn adversarial probing |
| Rigorous reproducible research-grade eval | Inspect AI | UK AISI-backed, most rigorous methodology |
| Real-time protection for live apps | NeMo Guardrails | Runtime guardrail (use with red team, not instead) |
Most teams' actual combination: promptfoo or Garak for red teaming + DeepEval for evaluation + NeMo Guardrails for live protection. You don't need all six—pick by your stage.
Six: Three Pitfalls
One: high stars don't mean right for you. promptfoo has the most stars (24k) but leans toward CI engineering scenarios; PyRIT has only 4,270★ yet is a powerhouse for multi-turn adversarial research. Pick by need, not by star count—stars reflect community heat, not your fit.
Two: guardrails and red teaming are two different things. NeMo Guardrails is a protection layer; installing it doesn't mean you've done red teaming. Guardrails block known patterns; red teaming finds unknown vulnerabilities. The full flow is red team → evaluate → guardrails → red team again. This site's red teaming SOP has the complete breakdown.
Three: without CI integration, it's a one-off. The biggest pitfall in red teaming is "run it once before launch and shelve it." Model iterations and prompt changes introduce new vulnerabilities—red teaming must run in CI on every release. promptfoo fits this best, DeepEval second. Pair it with the tool calling SOP to understand your agent's tool capability boundaries, so you know which attack surfaces to test.
Seven: FAQ
Q: What's the difference between red teaming and evaluation? A: Red teaming actively simulates attacks to find vulnerabilities ("can it be broken"); evaluation measures model quality and behavior ("how well does it answer"). promptfoo/PyRIT/Garak lean red team, DeepEval/Inspect AI lean evaluation—they're complementary, not mutually exclusive. A complete security practice needs both.
Q: I don't know Python—can I still do red teaming? A: Yes. promptfoo is written in TypeScript and config-file-driven—you can run red teaming without writing code. The other five all require Python. If your team is in the Node ecosystem, promptfoo is the only natural fit.
Q: Can NeMo Guardrails replace red teaming? A: No. Guardrails is a runtime layer that blocks known attack patterns; red teaming finds unknown vulnerabilities before launch. You still need red teaming even with guardrails installed, because the guardrails themselves can be bypassed. Use them together.
Q: PyRIT and Garak both lean red team—which one? A: Depends on whether you need multi-turn. PyRIT excels at multi-turn adversarial probing (escalating attacks step by step), suited for testing agentic applications; Garak excels at broad single-turn vulnerability scanning, suited for a pre-launch checkup. Testing agents, pick PyRIT; testing single-turn models, pick Garak.
Q: The EU AI Act mandates red teaming—which tool helps with compliance? A: The Act requires the process and documentation of "adversarial robustness testing," without mandating a specific tool. promptfoo (generates reports, CI-integrable, well-documented) and Inspect AI (rigorous methodology, reproducible) are best suited for producing compliance evidence. The exact format depends on what your legal team accepts.
References
- promptfoo (promptfoo/promptfoo) GitHub
- DeepEval (confident-ai/deepeval) GitHub
- Garak (NVIDIA/garak) GitHub
- NeMo Guardrails (NVIDIA-NeMo/Guardrails) GitHub
- PyRIT (microsoft/PyRIT) GitHub
- Inspect AI (UKGovernmentBEIS/inspect_ai) GitHub
- This site: OpenAI Astra pause hotspot ｜ AI Red Teaming SOP ｜ Agent Tool Calling SOP ｜ MCP Clients Comparison