PAGES
← all tools
◆ AI Red Teaming & Testing

FuzzyAI

open-source

FuzzyAI tests AI agents by fuzzing tool calls to find security vulnerabilities and unsafe behavior.

source ↗ rss ↗ What is a cyber toolchain?

Summary

FuzzyAI is an open-source LLMLLMA large language model, a neural network trained on large amounts of text to recognize patterns and generate language. Applications use it for tasks such as answering, summarizing and writing code. fuzzer that automates the search for jailbreaks and other security vulnerabilities in LLM APIs, using a range of attack techniques such as ManyShot, Taxonomy, and ArtPrompt against models hosted on OpenAI, Claude, Ollama, or a custom REST endpoint. It runs from the command line, installed via pip or Poetry, with an experimental web UI for those who prefer not to script against it. This puts it in the hands of developers and security researchers testing their own LLM integrations rather than SOCSOCA security operations center, the team or function that monitors threats, investigates alerts and coordinates incident response. Its work can be internal, outsourced or shared. teams monitoring live traffic, and it sits in the emerging red-teaming space for AI systems rather than traditional application security. First committed in 2024 with 12 contributors, the project has seen only 4 commits in the past year, suggesting slow, low-volume maintenance.

FuzzyAI tests AI agents by fuzzing tool calls to find security vulnerabilities and unsafe behavior.

How FuzzyAI helps

Which prompts can bypass the safety guardrails on our LLM endpoint before an attacker finds them?

runs a battery of attack techniques such as ManyShot, Taxonomy, and ArtPrompt against the model and reports which prompts produced a jailbreak

Does our system prompt hold up against known jailbreak patterns?

replays templated attacks, including many-shot and taxonomy-based variants, against the configured model and flags which ones got past the instructions

Is our custom REST API wrapper around a model as safe as the vendor's hosted endpoint?

points the same attack library at a custom REST endpoint instead of OpenAI, Claude, or Ollama, so results can be compared across integrations

Which attack technique is most effective against a specific model we've deployed?

runs each technique in turn against the target model and reports per-technique outcomes, showing which category of prompt manipulation succeeded

Can we catch a jailbreak regression before it reaches production?

reruns the same fuzzing session against a new model version or prompt revision and surfaces prompts that now succeed where they previously failed

What does a successful jailbreak against our model actually look like?

logs the full exchange for each attack attempt, so the winning prompt and the model's unsafe response can be inspected directly rather than inferred from a pass/fail score

all 5 features, with the evidence for each →

Languages

Not documented

Features

5 capabilities

Built from FuzzyAI's releases, docs, README and our own runs. “Seen” marks the month we first spotted a release the vendor didn't date. Group names and tags are our labels.

Compare with peers →

What FuzzyAI offers you

  • Command-line interface1
All capabilities 5 capabilities
Local model attack execution shipped Executes attacks using local LLMs via Ollama or displays available tool options. 2 other sources

Also counts toward: Command-line interface

example

  • Show fuzzer help and options fuzzyai fuzz -h
  • Default attack using llama over Ollama fuzzyai fuzz -m ollama/llama3.1 -a def -t "Harmful_Prompt"
Automated LLM vulnerability fuzzing documented Identifies jailbreaks and security vulnerabilities in LLM APIs using multiple attack techniques. 1 other source

docs

  • Automated LLM fuzzer that identifies jailbreaks and security vulnerabilities in LLM APIs using multiple attack techniques.
Custom API endpoint exploitation shipped Targets custom REST API endpoints using defined attack vectors. 1 other source

Shared capability: REST API

example

  • Attack a custom REST API endpoint OPENAI_API_KEYT=sk-3fa1... fuzzyai fuzz -a def -x openai/gpt-4 -e classifier_model=openai/gpt-4 -m rest/http.raw -e host=localhost -e port=8000 -e scheme=https
OpenAI-specific attack execution shipped Performs specialized attacks like ManyShot and Taxonomy against OpenAI interfaces. 1 other source

example

  • ManyShot and Taxonomy attacks via OpenAI OPENAI_API_KEY=sk-3fa1... fuzzyai fuzz -m openai/gpt-3.5-turbo -a man -a tax -t "Harmful_Prompt"
Prompt injection attacks shipped Executes prompt-based attacks such as ArtPrompt against target applications. 1 other source

example

  • ArtPrompt attack on prompts file with Claude ANTHROPIC_API_KEY=deadbeef... fuzzyai fuzz -m anthropic/claude-3-5-sonnet-20240620 -a art -T src/fuzzyai/resources/adv_prompts.txt -e blacklisted_words="bomb,garbage,acid,ugly,self-harm,kill your"

Lines in monospace are the tool's own words: help text parsed from its source, or an endpoint from its API document. Under release, a line is our summary of a dated release. Under docs, it is quoted from the tool's own documentation or website. Each links back to the source it came from.

Score card

  1. Shipping velocity releases a month, last 12 months
  2. Documentation quality release-notes rubric, 0–100
  3. Breaking-change discipline how rarely a release breaks users, and notice given
  4. Maintenance risk quiet time against its own cadence, maintainers
  5. Innovation & surface new capabilities, API and MCP surface
  6. Community health
  7. Contributor trend distinct committers, this year against last
  8. Maintainer concentration bus factor now against a year ago
  9. Issue backlog trend issues closed per issue opened, 12 months
  10. Responsiveness median time to a maintainer’s first reply

Nine scores for FuzzyAI, each built from the releases, notes and API history this site tracks — and, on Researcher and Business, where it stands among every measured tool in its category and how that has moved.

Score cards are part of every paid plan.

See plans →

Each score is 0–100 on a published rubric, derived from data this site already shows. A dimension we cannot measure says so — it is never scored 0 and never enters a rank.

How this is scored
  • Shipping velocity — releases a month over the last 12 months, on a log scale where 16 a month scores 100. Needs 3 releases in the window.
  • Documentation quality — the release-notes rubric published on the analytics page, unchanged.
  • Breaking-change discipline — the share of releases carrying a breaking change (none scores 100, half or more scores 0). Only projects that label breaking changes are measured: a clean record cannot be told from an unlabelled one.
  • Maintenance risk — quiet time measured against the tool's own release cadence (quiet longer than 90 days and three typical gaps scores 0), plus the number of active maintainers and whether contributors are growing, where known. Higher is lower risk.
  • Innovation & surface — new capabilities in the feature ledger this year, API and MCP operations added, and the share of commits that are features: the mean of whichever we hold.
  • Community health (open-source tools only) — each is a direction, not a size: contributors this year against last, the bus factor now against a year ago, and issues closed per issue opened, where steady scores 50, doubled 100 and halved 0; plus the median time to a maintainer's first reply (an hour scores about 100, a month or more 0). A commercial card has no repository and carries none of the four.
  • Documented delivery (commercial tools) — when a vendor's documented new capabilities have slowed against its own baseline, that counts against the overall standing as heavily as two low dimensions. Steady delivery, or too little history to judge, leaves the standing alone; it is never a sixth score.
  • Open-source tools are scored on repository signals; commercial tools on their published changelog. Ranks compare tools within one category and one profile and need at least five measured peers; the overall standing needs three ranked dimensions. Best-in-breed or consolidator is a shape, not a quality, and is never scored.

Release-note detail over time

See how much detail FuzzyAI puts in its release notes, quarter by quarter, and compare it with tracked tools in the same category. This history uses up to ten retained, reviewed notes per tool; sparse or blank older quarters may reflect that limit.

Checking release-note history…

Explore the full chart →

Releases

Nothing shipped since we started watching. Releases and docs updates land here as they happen.

watching since Jul 18, 2026

Project health

Issue, contributor and maintainer activity in cyberark/FuzzyAI, 2023-10 – 2026-09, as of 2026-09-25. Twelve-month figures first; the trend lines run the whole window.

  1. Contributors 2 −9 vs prior 12 months
  2. Maintainers 0 +1 joined · −4 left, last 12 months
  3. Bus factor 0 committers behind half the commits
  4. Community-filed 100% of issues opened, last 12 months
  5. Median time to close < 1 day first maintainer reply 20 hours
  6. Open backlog 4 of 20 issues ever filed
  1. Issues opened / month 0 · 0 communitymaintainers
  2. Open backlog 4
  3. Contributors / month 0
  4. Commits / month 0
  5. Stars 1,574
Issue themes, last 12 months
  • other 100%

Counts from the project's public repository and git history; no individuals are identified. Maintainers are the committers covering 80% of the last year's commits. how we measure this

Libraries its release ships

What its latest release locks, checked against known advisories.

We can only check tools that publish GitHub releases.

Add the badge to your README

Maintain this project? The badge links readers straight to this page, so they can follow every release here.

Tracked by Cyber Toolchain Choose a badge style and copy the code →
compare.md

    Free the public facts, no sign-in Any paid plan all other sections Researcher & Business pricing history and category rank

    my-toolchain — 0 tools

      Betas, release candidates and previews. Turn it off to see final releases only. It also applies to changes you ask for over MCP when you're signed in.
      browse all tools → score card for your toolchain →
      cyber-toolchain / feedback

      Send us feedback