Project summary

Provael is an open-source red-team for robot policies. It attacks a vision-language-action policy in simulation and returns an attack-success rate with a benign control, a 95% interval, and an evidence pack (SARIF, OSCAL, CycloneDX ML-BOM). Apache-2.0, runs on CPU, on PyPI.

What is measured today: one reframed instruction drove a real SmolVLA policy off its benign task on 44 of 50 matched pairs, across all ten libero_object tasks in LIBERO. A benign reword of the same instruction did it 1 time in 50. Nonsense text: 0 in 50. No attack at all: 3 in 50. McNemar p = 2.3e-13. All four arms ran inside one report, so every comparison is paired on matched (task, seed) cells instead of stitched across runs.

The gap I want to close: 15 adversarial attack families are registered and unit-tested on CPU. Only 3 have ever been run against a real policy. 12 are stub-validated only, and there are zero hardware results. That split is printed on the site next to the good number, because registered is not measured, and I do not want anyone reading it as coverage.

Publishing the gap does not fix it. Running the families does, and that costs GPU time I have been paying for out of pocket.

What are this project's goals? How will you achieve them?

The goal is narrow: move attack families from "registered" to "measured against a real policy", and publish whatever comes back, including the nulls.

The runner already exists. examples/gpu-ci/modal_libero_suite.py runs the suite on a rented GPU in cost-ordered stages, sharding one task per container. Sharding is a survivability decision, not a speed one: Proval attack cannot resume, so a 25-hour single container that dies at hour 19 loses nineteen hours. Ten containers lose one task.

The cost table in that file is measured, not projected. A full ten-task screen is 400 episodes, 15.4 GPU-hours, $12.29 actual. The estimate before it ran was $10.17, so the estimate was 21% low, and the table says so. The control arm is 200 episodes, about $6.15.

Each shard writes its own report.json, which is the attestable artifact: signable, digestible, and pinned into the public evidence manifest the website builds from. If a family does not transfer, that null publishes the same way the 44/50 did. Two of the eight attacks in the run I already published came back 0 out of 50, and both are on the site.

How will this funding be used?

All of it on GPU compute. No salary, no hardware, no travel.

At $12.29 per full ten-task screen and about $6.15 per control arm, $500 is roughly 27 screens run with their controls, or about 40 without. I would rather run fewer with controls. A result without its benign arm is the exact thing I spend most of my time telling people not to trust, so I am not going to buy more of it.

Order I would run them in:

1. The 12 stub-validated families against SmolVLA, so the 3-of-15 number moves.

2. Whichever ones transfer, against a second policy family (OpenVLA or pi0). One policy is not transfer.

3. Whatever is left on Meta-World, since everything measured today is LIBERO.

If it runs out partway, the partial result still publishes. No milestone has to be completed for the money to have bought something.

Who is on your team? What's your track record on similar projects?

One person. Me, built alongside a full-time job.

Three things worth knowing:

1. I built pyAGI, an autonomous-agent Python framework, acquired in 2025 by Kyle Morris (co-founder, banana.dev) and Jeffrey. I have shipped a developer tool that someone else wanted to own.

2. Six-plus years shipping production AI as a GenAI Architect and Tech Lead across fintech, AI-training, and GenAI companies. Multi-agent systems, LLM infrastructure, observability and governance, with real users.

3. Provael: 38 releases since 7 June 2026, 87 test files, 5 policy adapters, 6 simulator suites, three compliance emitters. 4,462 lifetime PyPI installs against 5 GitHub stars, which I read as CI and scanners more than people. I would rather say that than quote the 15,261 mirror-inclusive number.

The part I would actually want a funder to check: the marketing site fails its own build if a published number drifts from the pinned artifact in the tool's repo. Eleven build-time checks, and most of them exist because a specific wrong claim shipped green first. There is an errata page listing the ones I got wrong. That is the closest thing I have to a track record on honesty, and it is checkable without talking to me.

What are the most likely causes and outcomes if this project fails?

Most likely cause: the families do not transfer, and most of the 12 come back at 0 out of 50. That is a real outcome and I would publish it. It makes the tool more honest and the company weaker, and I would much rather find it with $500 than with a customer.

Second, and bigger than this grant. I cannot find a single public example of any notified body, TUV, UL, SGS or Intertek citing a named third-party AI red-teaming tool in a conformity assessment. The whole compliance thesis rests on simulation-based attack evidence being admissible in a technical file, and it has zero verified precedent. One phone call closes that question and I have not made it yet.

Third, the boring one. I work full-time elsewhere and have not completed IP or employment clearance for this. The repository is public and Apache-2.0 and built on my own time, but that position is not papered. Cheaper to say it now than to have it surface later.

If it fails, the code stays up under Apache-2.0, and the measurements stay published. Nobody has to take my word for a number in it, which is the part I am most confident about.

How much money have you raised in the last 12 months, and from where?

Nothing. No pre-seed, no angel, no grant, no debt, no convertible instrument. Zero outside capital, zero revenue. Self-funded out of pocket alongside a job.

One pending item for completeness: I applied to Emergent Ventures in July 2026 for a $50,000 research grant to extend the measured result across more open policy families. No decision as of today. Nothing received, nothing committed.