Project summary

Tools can give an AI agent technical access to an API without anyone actually delegating the business choice to it. Worse, getting back a 200 OK from an endpoint doesn't mean the database actually updated. I'm building a simple, runnable open-source testbench to catch these exact discrepancies across human sign-offs, agent handoffs, tool calls, and actual system states. A $6k starter build gives us 6 core test cases; $16k lets us expand to 18 cases, build 2 adapters, and get an independent technical critique.

What are this project's goals? How will you achieve them?

I want to create a practical, open reference suite for safety researchers and engineers who need to answer a straightforward question: when an agent executes a tool call or hands off a task, who actually authorized the decision, and can we independently verify what changed on the server?

Most trace tools assume a valid API token implies human approval, or that a success log proves an action finished. This testbench intentionally breaks those assumptions.

Here is how I plan to build it:

1. Synthetic Case Schema: I'm defining a lightweight case format that keeps separate, isolated records for the user request, policy rules, active technical grants, human approval steps, tool calls, and actual database/state readbacks. If an evidence source is missing, the evaluator outputs UNKNOWN rather than guessing.

2. Replayable Fixtures & Runner: I'll release a zero-customer-data test runner with synthetic test cases. These will cover real-world edge cases like overbroad permissions, bypassed approvals, expired tokens, scope creep during multi-agent handoffs, silent API write failures, and missing telemetry. Instead of trusting the agent's narrative, the runner uses independent before-and-after sandbox snapshots as the ground truth.

3. Expanded Suite & Outside Review: At the $16k tier, I'll expand the test suite to 18 scenarios across two integration patterns, add failure-injection tests, and pay an outside engineer to tear apart our fixture assumptions and write an honest critique of our blind spots.

Everything will be published under a free Apache 2.0 open-source license. It's built to run locally out of the box without requiring paid subscriptions, proprietary platforms, or specialized enterprise setups.

How will you know it worked?

At the $6,000 minimum:

- We publish a public GitHub repository with a single, repeatable command to run the suite locally.

- Six complete synthetic cases run deterministically, correctly catching missing approvals, scope changes, and unapplied state changes.

- Clear CLI outputs (SUPPORTED / NOT SUPPORTED / UNKNOWN) paired with a straightforward limitations document.

At the $16,000 target:

- At least 18 versioned test cases covering single-agent, human-in-the-loop, and multi-agent delegation paths.

- Two working integration adapters and a mutation regression test suite.

- A published, unedited technical critique from an outside reviewer detailing where our testbench succeeds and where its assumptions break down.

How will this funding be used?

I am requesting a $16,000 target with a $6,000 minimum. The public $6,000 starter release would pay for 50 hours of original research and assurance protocol work ($3,000), 35 hours of standalone testbench engineering ($2,275), synthetic compute/test infrastructure ($425), and documentation ($300).

At the full $16,000 goal: $6,600 for 110 hours of protocol/research work; $5,200 for 80 hours of open reference implementation; $2,400 for an independent 24-hour technical critique; $800 for isolated experiments/infrastructure; and $1,000 for reproducibility documentation, accessibility and public release. These are proposed direct project costs, not contracts or salaries already paid. No retroactive work, marketing, investor fundraising, or commercial HAIEC roadmap features are included.

Delivery would begin after Manifund approves and releases funds: six-case minimum in approximately 4–6 weeks; full tier 18-case suite and report in 8–10 weeks. The publication and licensing boundary will be confirmed under the actual grant agreement before receiving any funds.

Who is on your team? What's your track record on similar projects?

I am Subodh KC, founder of HAIEC (the Human AI Evidence Company) and author of the ISAF research program. My prior work includes an author-issued ISAF v1 preprint (DOI 10.5281/zenodo.18080355), a newer unpublished ISAF v2 study of authority and evidence sufficiency, and open technical packages for AI application scanning, MCP tenant isolation, and model security signals. I previously served as a Fortune 100 AI Strategy Core Team Lead through a contractor engagement.

HAIEC's AI Action Path Assurance work received a MunichTech EXPO 2026 Grand Challenge award on Devpost. Our team also participated in TM Forum Innovate Americas 2026, where we produced bounded telecom simulation artifacts and a public evidence archive. These establish development experience, not independent proof that every HAIEC control works in production.

Existing contributors include full-time unpaid operations/project support and part-time technical advisors; specific new grant-funded roles will be contracted only if funded. We have separately implemented a verified ChatGPT/KestrelVoice MCP connection for customer onboarding and approved receptionist configuration. That product is a prior example of consequential agent workflows, but this grant's deliverables will be independent, fictional/synthetic, and publicly reproducible without customer data.

What are the most likely causes and outcomes if this project fails?

Risk 1: The synthetic cases may be too simple to transfer to real agent systems. We will document that limitation, include handoff/approval/missing-evidence edge cases, and invite outside critique rather than claiming industry-level accuracy.

Risk 2: A replayable action log may be mistaken for proof of what happened. The oracle will use a separate sandbox state record with source identity, evaluation time, and before/after data. Even that proves only the sandbox outcome, not universal real-world guarantees.

Risk 3: Source, delegated permission, and policy rules may be ambiguous. We will use explicit contracts, qualified fields, and UNKNOWN/NOT_ASSESSED when evidence is missing. Counterexamples will be retained in the public report.

Risk 4: Funding or reviewer availability may be insufficient. The $6,000 starter tier stands alone without needing the expanded suite or an already-committed reviewer. At the higher tier, independent critique is a deliverable, subject to recruitment and conflict checks.

Risk 5: Duplication with other AI-agent benchmark projects. Existing work already provides policy checkers and trace monitors. This project deliberately focuses on the missing junction between business-level delegation, multiple agent/tool handoffs, and independently established external effect. We will credit and reuse competent baselines where licenses permit.

How much money have you raised in the last 12 months, and from where?

FOUNDER VERIFICATION REQUIRED BEFORE PUBLICATION: No money has been awarded under this Manifund project. I have prepared applications seeking support for separate research questions, including a $45,000 ISAF independent evaluation proposal for TAIF and a $49,000 restricted-evidence research proposal for Foresight; neither is confirmed funded. I will disclose any actual cash grants, donations, investments, or other funding relevant to this form after reviewing the past-12-month funding ledger. I will update Manifund and adjust costs if a future award overlaps with this project's billable work. Pending applications are not counted as funding raised.