Project summary

An API permission isn't the same thing as a human decision, and getting a 200 OK back from a server doesn't mean your write actually landed. When Subodh and I were going over our recent agent logs, we kept bumping into this exact issue: system logs show success, but nothing actually changed in the background. I'm building a simple, open-source testbench to make those edge cases testable. A $6,000 starter build gives us 6 core test scenarios. $16,000 lets us expand to 18 cases, hook up 2 adapters, and pay an outside researcher to tear apart our assumptions.

What are this project's goals? How will you achieve them?

I want to put out a practical, no-nonsense reference tool for AI safety researchers and engineers. The core question we're answering is straightforward: when an agent calls an MCP tool or passes off a task to another agent, who actually delegated that decision, and can we prove what changed on the server without relying on the agent's own output?

Right now, most trace tools make two big assumptions: they treat a valid API token as proof of human sign-off, and they take an HTTP 200 response as proof of a completed action. We're building this testbench specifically to break those assumptions.

Here is the plan:

First, I'll write a clean synthetic case schema. It keeps isolated records for the original user prompt, policy boundaries, active technical grants, human approvals, tool calls, and actual database readbacks. If evidence is missing anywhere along the chain, the system outputs UNKNOWN instead of making a guess.

Second, I'll release a zero-customer-data test runner with pre-built fixtures. We'll cover real failure modes: overbroad permissions, bypassed approvals, expired tokens, scope creep during multi-agent handoffs, silent write drops, and missing telemetry logs. Instead of trusting the agent's narrative, the runner checks independent before-and-after sandbox snapshots.

Third, at the $16k target, we'll expand the suite to 18 scenarios across two integration styles, add mutation/failure regression tests, and pay an independent reviewer to find our blind spots and write an unedited critique.

We're publishing everything under Apache 2.0. The testbench runs locally on a standard machine with zero paid API subscriptions or proprietary accounts required.

How will you know it worked?

For the $6,000 minimum:

- We push a public GitHub repo with a pinned setup that runs locally via a single CLI command.

- Six synthetic scenarios run deterministically, correctly flagging unapproved actions, scope changes, and silent write failures.

- The tool outputs clear status codes (SUPPORTED / NOT SUPPORTED / UNKNOWN) alongside a plain-language limitations doc.

For the $16,000 target:

- 18 complete test cases covering single-agent, human-in-the-loop, and multi-agent delegation paths.

- Two integration adapters plus a seeded mutation regression test suite.

- A published, unedited method critique from an outside reviewer highlighting where our setup works and where it falls short.

How will this funding be used?

I am requesting a $16,000 target with a $6,000 minimum. The public $6,000 starter release would pay for 50 hours of original research and assurance protocol work ($3,000), 35 hours of standalone testbench engineering ($2,275), synthetic compute/test infrastructure ($425), and documentation ($300).

At the full $16,000 goal: $6,600 for 110 hours of protocol/research work; $5,200 for 80 hours of open reference implementation; $2,400 for an independent 24-hour technical critique; $800 for isolated experiments/infrastructure; and $1,000 for reproducibility documentation, accessibility and public release. These are proposed direct project costs, not contracts or salaries already paid. No retroactive work, marketing, investor fundraising, or commercial HAIEC roadmap features are included.

Delivery would begin after Manifund approves and releases funds: six-case minimum in approximately 4–6 weeks; full tier 18-case suite and report in 8–10 weeks. The publication and licensing boundary will be confirmed under the actual grant agreement before receiving any funds.

Who is on your team? What's your track record on similar projects?

I am Subodh KC, founder of HAIEC (the Human AI Evidence Company) and author of the ISAF research program. My prior work includes an author-issued ISAF v1 preprint (DOI 10.5281/zenodo.18080355), a newer unpublished ISAF v2 study of authority and evidence sufficiency, and open technical packages for AI application scanning, MCP tenant isolation, and model security signals. I previously served as a Fortune 100 AI Strategy Core Team Lead through a contractor engagement.

HAIEC's AI Action Path Assurance work received a MunichTech EXPO 2026 Grand Challenge award on Devpost. Our team also participated in TM Forum Innovate Americas 2026, where we produced bounded telecom simulation artifacts and a public evidence archive. These establish development experience, not independent proof that every HAIEC control works in production.

Existing contributors include full-time unpaid operations/project support and part-time technical advisors; specific new grant-funded roles will be contracted only if funded. We have separately implemented a verified ChatGPT/KestrelVoice MCP connection for customer onboarding and approved receptionist configuration. That product is a prior example of consequential agent workflows, but this grant's deliverables will be independent, fictional/synthetic, and publicly reproducible without customer data.

What are the most likely causes and outcomes if this project fails?

Risk 1: The synthetic cases may be too simple to transfer to real agent systems. We will document that limitation, include handoff/approval/missing-evidence edge cases, and invite outside critique rather than claiming industry-level accuracy.

Risk 2: A replayable action log may be mistaken for proof of what happened. The oracle will use a separate sandbox state record with source identity, evaluation time, and before/after data. Even that proves only the sandbox outcome, not universal real-world guarantees.

Risk 3: Source, delegated permission, and policy rules may be ambiguous. We will use explicit contracts, qualified fields, and UNKNOWN/NOT_ASSESSED when evidence is missing. Counterexamples will be retained in the public report.

Risk 4: Funding or reviewer availability may be insufficient. The $6,000 starter tier stands alone without needing the expanded suite or an already-committed reviewer. At the higher tier, independent critique is a deliverable, subject to recruitment and conflict checks.

Risk 5: Duplication with other AI-agent benchmark projects. Existing work already provides policy checkers and trace monitors. This project deliberately focuses on the missing junction between business-level delegation, multiple agent/tool handoffs, and independently established external effect. We will credit and reuse competent baselines where licenses permit.

How much money have you raised in the last 12 months, and from where?

FOUNDER VERIFICATION REQUIRED BEFORE PUBLICATION: No money has been awarded under this Manifund project. I have prepared applications seeking support for separate research questions, including a $45,000 ISAF independent evaluation proposal for TAIF and a $49,000 restricted-evidence research proposal for Foresight; neither is confirmed funded. I will disclose any actual cash grants, donations, investments, or other funding relevant to this form after reviewing the past-12-month funding ledger. I will update Manifund and adjust costs if a future award overlaps with this project's billable work. Pending applications are not counted as funding raised.