10 Open-Source Security Tools to Build and Red-Team Safe AI Systems

On October 7, 2026, Leonard Rodman (@RodmanAi) highlighted ten essential open-source security repositories designed to assess vulnerabilities, execute automated red-teaming, and construct safer systems for large language models and generative AI applications. As AI deployments expand, addressing security challenges such as prompt injection, jailbreaking, and sensitive data leakage has become an operational priority. This curated guide breaks down the ten open-source toolsets shared by Rodman, helping engineering teams explore and integrate them across their systems.

Image source: @RodmanAi on X

Securing modern generative AI systems requires more than simple system prompt instructions. Engineering teams can adopt a layered approach combining vulnerability scanning, multi-turn red-teaming simulations, and runtime guardrails. Below is an overview of the ten open-source repositories, categorized into offensive vulnerability probing, real-time runtime defenses, and model reliability verification.

Vulnerability Scanning and Prompt Injection Defense Tools (1–4)

During early development and evaluation cycles, offensive security tools are critical for discovering how foundation models respond to adversarial scenarios before production deployment.

1. Garak (Generative AI Red-teaming & Assessment Kit): An open-source LLM vulnerability scanner developed under NVIDIA. Functioning much like nmap or Metasploit for large language models, Garak systematically probes conversational systems and foundation models for hallucination, prompt injection, data leakage, toxic output, and jailbreak weaknesses. It incorporates static, dynamic, and adaptive probes, including cutting-edge multi-turn techniques like GOAT (Generative Offensive Agent Tester), which uses an adversarial LLM to iteratively formulate attacks via the Observation-Thought-Strategy-Response (O-T-S-R) reasoning framework.
2. PyRIT (Python Risk Identification Toolkit): Microsoft AI Red Team's open-source framework designed for scalable, automated, and human-led red teaming across generative AI systems. PyRIT enables security operators to execute advanced multi-turn attack strategies—such as Crescendo, TAP, and Skeleton Key—with minimal setup. Its composable scenario framework supports standardized evaluations spanning content harms, data exfiltration, and psychosocial risks, offering model- and platform-agnostic testing across single- and multi-modal architectures.
3. Promptfoo: A security evaluation tool designed to evaluate prompts, models, and AI applications for security issues. It enables teams to test systems against critical security concerns such as prompt injection and OWASP LLM Top 10 vulnerabilities, ensuring potential security flaws are evaluated early.
4. Rebuff: An open-source framework designed to detect and defend against prompt injection attacks. It focuses on identifying and intercepting adversarial injection attempts that seek to subvert system instructions and compromise AI applications.

Runtime Guardrails and Real-Time Threat Detection Layers (5–7)

When applications handle live user interactions, security controls are required to inspect inputs and outputs continuously and enforce behavioral policies.

5. LLM Guard: A security toolkit designed to add security checks and safeguards to LLM applications. It enables teams to integrate security checks and protective guardrails across application pipelines to mitigate operational risks.
6. Vigil: An open-source security tool tailored to scan LLM inputs and outputs for potential security threats. It inspects prompts and responses to detect and flag emerging risks across the application boundary.
7. NeMo Guardrails: An open-source framework designed to add programmable safety and control layers to AI applications. It allows developers to define programmable control layers to ensure conversational and generative systems remain within established safety guidelines.

Model Reliability Evaluation and Automated Fuzzing Frameworks (8–10)

Long-term reliability requires continuous testing to evaluate model risks and identify unexpected edge-case failure modes.

8. Giskard: An open-source testing platform built to test and evaluate AI models for reliability and security risks. It provides structured testing to assess potential model vulnerabilities and verify operational reliability before deployment.
9. DeepTeam: An open-source tool built to red-team LLM applications and test their security. It evaluates application defenses against adversarial scenarios and helps teams identify vulnerabilities before production use.
10. FuzzyAI: A security testing tool designed to test AI systems for vulnerabilities using automated fuzzing. By running automated fuzzing tests, it helps uncover hidden weaknesses and evaluate system robustness against unexpected adversarial inputs.

Original source

• Leonard Rodman Official X (@RodmanAi): 10 GitHub Repos to Build and Test AI Security Systems
• NVIDIA Garak Official Repository: GitHub - NVIDIA/garak
• Microsoft PyRIT Official Repository: GitHub - microsoft/PyRIT