You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$50,000 left to reach the funding goal.
Argus is an open-source project that runs red team tests for AI agents. It is for both black-box and open-source agents and was released under Apache 2.0.
Argus has 247 probes based on OWASP LLM Top 10, MITRE ATLAS etc. Including the most popular attack vectors like Direct/Indirect Prompt Injection, Confused Deputy, Shadow MCP, MCP Poisoning, and so on. It supports 8 different target adapters. The reports in SARIF or HTML.
I built Argus because the safety of agent builders is becoming increasingly important nowadays. Argus can help someone run a consistent set of probes before rollout and inspect the evidence behind each result.
Over the next 12 months, from Feb 2027 to Jan 2028, I want to address a gap in Argus’s current coverage. It tests individual agents, but it does not yet adequately test attacks that move through several steps or across multiple agents.
Months 1–6:
I’ll build a probe family for orchestration workflows. The tests will follow malicious instructions planted in an earlier step and check whether they cause code execution or data exfiltration later in the workflow. I’ll also run and publish a benchmark against open-source orchestration targets.
Months 4–9:
I’ll publish a versioned evidence corpus containing SARIF results and machine-readable summaries. It will include a rerun harness so that someone reviewing a finding can repeat the scenario and check the verdict themselves.
Months 6–12:
I’ll build a dashboard where users can see security scores, review remediation plans and follow an attack path through a graph. They will also be able to select a real-world environment for their security tests.
By month 6, the orchestration probes should be merged and the public benchmark results released. By month 9, the evidence corpus and rerun harness should be available, with enough information for third parties to repeat every scenario and obtain the same verdicts.
Shipping those components is only part of the test. I also want at least 2–3 independent agent teams to run Argus, and external users to run the public benchmark within its first two months. My own runs won’t count toward that adoption target.
By month 12, I’ll consider the project successful if the v2 probe set is released and builders are using its reports when deciding whether an agent is ready to deploy.
The total budget is $50,000. Of that, $38,000 covers part-time builder work over 12 months, $8,000 covers compute and LLM API costs for probes and benchmark runs, and $4,000 covers a community validation sprint and incidentals.
If I raise only the $20,000 minimum, I’ll limit the work to the first phase: the orchestration probe family and one public benchmark run.
The team is Seth Porter and me. Seth is a classmate from the BlueDot AI Safety course and will be responsible for data analysis.
I’m Yang Gao, a Staff Security/QA Engineer at Fortinet in the San Francisco Bay Area. I have about 18 years of experience in cloud security, network security and test automation. I created Argus and maintain its probe model, target adapters and repository.
Argus itself is the clearest example of my work. It has 247 shipped probes, 8 target adapters and about 206 GitHub stars, and is available under Apache 2.0.
I also published a controlled comparison of two guards. In that experiment, a keyword guard let 22.9% of attacks through. An ML injection classifier reduced that figure to 2.9%, but also blocked 44% of benign prompts. I want builders to be able to measure that kind of trade-off before choosing a guard for deployment.
My paper, “How Reliable Is Your Jailbreak Judge? Calibration and Adversarial Robustness of Automated ASR Scoring (https://arxiv.org/abs/2606.25487),” examines the reliability of LLM judges, including the kind used to score some Argus probes.
I think adoption is the biggest risk. I could finish the probes and evidence corpus, but still fail to get builders to use them. Integration might take too much effort, or the reports might not answer the questions builders face when deciding whether to deploy.
I’ll look for that problem early. No external benchmark runs in the first two months would be a warning sign. So would validation calls where builders describe the reports as “interesting” but cannot identify a decision the findings would change.
If that happens, I’ll pause further expansion and work directly with 2–3 builders. I’ll adapt the reports to their deployment decisions, even if that requires reducing the scope.
The other substantial risk is maintenance. Agent frameworks change quickly, and keeping probes working could consume the builder-time budget. The rerun harness will help expose regressions as they occur, although maintaining the affected probes will still require time.
Argus received $5,000 in Azure credits during the last 12 months. I have covered the remaining costs myself.
Three other grant applications for this same 12-month plan are currently under review. If any receives funding, I’ll update this page and revise the budget and scope so that the same work is not funded twice
There are no bids on this project.