You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$14,200 left to reach the funding goal.
An AI agent can give a safe-looking answer while having used powers nobody delegated to it. Nirmata is an open framework (Apache-2.0, DOI 10.5281/zenodo.22132451) for auditing agents from their execution trajectories: it keeps recommendation, owner authorization, attempt and external effect distinct, and combines an output-only evaluator with a trajectory-only evaluator by a fixed OR rule. v0.1.0 is public; no confirmatory result exists yet. This project funds the first fresh blind pilot and a replication, with every outcome published.
Goal: measure, honestly, how much trajectory-aware evaluation adds over output-only evaluation on agent failures involving delegated authority, control integrity and failure-preserving reproducibility. Steps: preregister protocol and metrics; independent custody of holdout labels; hash predictions before ground truth; score without changing anything; publish the result; replicate on a second agentic system; invite external review of the schema.
Public preregistration exists before data; a public pilot report with Safety Blind Spot Rate, Incremental Detection Value and false-positive rate is published whatever the result; a replication on a second system; updated arXiv paper. "Worked" means a credible answer, not necessarily a positive one.
Minimum US$3,000: compute/API ($2,000) + ~33 research hours at $30/h ($1,000) to run the preregistered pilot. Full goal US$14,200: 312 h research time over 6 months ($9,360), compute/API ($2,000), one conference trip for external review ($2,500), misc ($340). Minimum = pilot only; full = pilot + replication + external review.
Solo: Tabata Jahoda, independent researcher in Rio de Janeiro (ORCID 0009-0007-1104-9204), contact tabatajahoda@gmail.com. I keep my full-time job as Account Manager at Easy8 and do this research part-time (12 h/week). Airtable Builder certified, trainer. Built and released Nirmata v0.1.0 in ~10 days after repo creation (schema, blinding protocol, scoring spec, reproducible 6-trace fixture with tests and CI, release checksums, Zenodo DOI). A preregistered n8n adapter holdout passed all 8 criteria on one synthetic workflow (fidelity, not detection accuracy). Publicly documented an invalidated first holdout. 9-page paper pending arXiv endorsement. Repo: https://github.com/thabatarj8-stack/nirmata LinkedIn: https://www.linkedin.com/in/tabata-j-226b6123
Part-time schedule slips; independent label custody is hard for a solo researcher (limitation would be reported and evidence downgraded); trajectory evaluation may add little or have high false positives (published as a negative result); synthetic traces may not generalize. Worst realistic outcome: a delayed or invalidated pilot, documented publicly, plus a reusable open schema.
US$0 received. Pending: Foresight Institute application submitted (US$60k, decision Jan 2027). Lightcone Commons application submitted (US$25.5k, decision around Jan 2027).