The first public benchmark measuring whether deployed LLM systems stay inside their domain — built from a production gateway that survived 2,002 adversarial att
## The problem
Companies deploy LLMs inside their own subject matter, where users trust them most, but there is no public way to measure whether an output-containment layer actually keeps a model in scope. Buyers and regulators rely on vendor marketing.
## What we have built
ATRAKTA is a multi-tenant output-containment gateway in production since 2025. It keeps LLM answers inside each client's scope and reconducts drifting answers instead of only blocking them. We maintain a private adversarial battery (multi-turn and large-prompt "Quimera" attacks), preregistered lab protocols, sealed run logs, and we publish negative results, including a steering round that showed influence without factual gain. In our own remote benchmark, ATRAKTA and a strong baseline both contained nearly all attacks with overlapping confidence intervals, which is exactly why an open, neutral benchmark is needed. NVIDIA Inception member. atrakta.ia.br
## What this funding releases
1. Expand the private battery into a public, versioned set of adversarial cases
2. Open scoring harness and replication kit with sealed logs
3. Public leaderboard, with first external teams invited
4. Report both false blocks and missed attacks, with confidence intervals
## Budget
US$5,000 to US$8,000 for dataset expansion and the scoring harness. Solo founder with a Brazilian cost base.
## Why it matters
It makes the safety layer of deployed LLMs publicly measurable, from Brazil, a region the AI safety ecosystem barely covers.
Comments
No comments yet. Sign in to create one!