Project summary

Wolf Defender is our open-weight prompt-injection detection model, with around >10k monthly downloads (base + small) and usage in Apache Camel and Red Hat's TrustyAI.
We want to build the next generation of the model and making it able to protect multi turn agent workflows. For that we built upon the mmBERT encoder architecture and added swappable turn-memory and reading and writing to such memory. The encoder only ever sees the current turn, making this approach fast, RAM-efficient and reliable for progressive and adaptive prompt injections.

We are seeking 25,000$ to fund a focused development, data generation and evaluation phase. With a minimum of 5,000$ we could get enough GPU compute to initially train this approach on enough data and also do explorative red teaming work against the model.

The full budget would additionally support research into more sophisticated memory and RL approaches for such.

https://huggingface.co/patronus-studio/wolf-defender-prompt-injection

What are this project's goals? How will you achieve them?

Wolf Defender started with the premise to make a up-to-date prompt injection model with low false positive rate, which can be run on a CPU without exploding memory or latency. We achieved that in combination of our own ORT runtime. Memory is stable no matter of context length, speed up to 3k tok/s.
Now it is time to make Wolf Defender reliable in the agent world. The goals stay the same:
- high recall on actual threats
- low false positive rates
- multilingual
- able to understand progressive threats, authority and policy claims.

For that we have to solve a few problems. Security evaluations should not contain the current turn and all previous turns in their context (an approach currently used by big tech, e.g claude code). It would enable attention to all turns, but we can not do that reliable fast in realtime on a CPU or edge GPU. Also it would mean to re-encode every previous turn on every step. Thats why we use external memory tokens. See them as a security memory checkpoint containing the relevant stuff of previous turns. A writer writes from the encoder representations into the memory; a reader will read them.
Whats a good security memory? Good question. Thats something we want to find out with the funding. Our first tests showed this approach is able to learn on never-seen progressive attack benchmarks while maintaining stable false positive rates. But the best approach of writing and hierarchically storing memory is still open.
The last problem is amount of training data. We currently have ~100k. We aim to synthetically expand to ~500k.

How will you know it worked?

Our current evaluation consists of ~300k held-out data points of various public single and multi turn agent security benchmarks. Success means: No regression in single turn detection and a reliable multi turn model for progressive prompt injections, agent hijacking and other threats. Also we aim to work together with companies that provide non-public benchmarks.
Multi Turn detection will be measured with the StepShield approach, meaning false positives in an early turn (before the progressive attack finished) do not count as a detection success.

Additionally we can test our approach on various CPUs to measure speed and RAM usage. We aim to analyse 256 tokens in <100ms and 2048 tokens in <700ms.

How will this funding be used?

Our funding goal is 25,000$:

• 3,000$ for research and engineering time: developing evaluation workflows, improving training data, implementing multi-turn methods and preparing releases.

• 15,000$ for GPU compute: fine-tuning, controlled comparisons and training experiments.

• 5,000$ for adversarial evaluation: model API access and compute for generating and testing adaptive attacks.

• 2,000$ contingency for additional compute and experiments needed to investigate failures.

Our minimum funding requirement is 5,000$:

• 4,000$ for GPU compute.

• 500$ for adversarial evaluation.

• 500$ contingency.
  

Who is on your team? What's your track record on similar projects?

• Benedikt Veith, co-founder and CTO of Casdo Labs GmbH, the company behind Patronus Protect. He would lead this project.

  He has more than 12 years of software engineering experience and hands-on experience training, evaluating and deploying small security models. Currently finishing his AI and Data Science Bachelor

• Dominik Hommer, co-founder and CEO of Casdo Labs GmbH. Research and practical experience in ML. Bachelor in AI and Data Science.

Our existing Wolf Defender models receive over 10,000 downloads per month and are used in the Apache Camel and Red Hat ecosystems. This gives us an existing deployment base and experience preparing models for integration by other developers.

What are the most likely causes and outcomes if this project fails?

The main technical risk is that incorporating more context increases false positives or inference costs without reliably improving attack detection. Adaptive attackers may also expose weaknesses that cannot be resolved within this project's budget. First tests counter argue with the first risk.

If the multi-turn approach does not succeed, we would still aim to publish our results. Also all training intrinsically improves the Wolf Defender base model. So worst case is we get an even better version of that.

How much money have you raised in the last 12 months, and from where?

none; we are bootstrapped. Happy to connect