You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$50,000 left to reach the funding goal.
My research started with a practical security problem. An AI-generated cloud policy can look valid and still grant more access than the user requested. The same problem can appear elsewhere: an AI recommendation may sound convincing while using the wrong source, period, entity, or authority.
I want to build a small external verifier that checks whether an AI agent stayed within the task and limits it was given before an important action is approved. The verifier will use recorded evidence, calculations, tool outputs, and action boundaries. It will report whether a requirement passed, failed, or could not be decided. A failed requirement can stop the action. An unclear case can be sent to a human reviewer.
The project builds on IAMRAGBench, my existing public benchmark for security testing of LLM-generated cloud policies. I will first test the verifier on cloud-access tasks, where the allowed and forbidden actions can be measured precisely. I will then test a smaller set of public or synthetic financial-evidence tasks to see which parts transfer beyond IAM.
The project will produce an open-source prototype, reusable test cases, comparisons with ordinary validation and LLM-as-judge, and clear reproduction instructions.
Because this is a public proposal, I am not publishing detailed implementation choices, unpublished attack recipes, or partner-specific information here. The final research release will contain enough technical detail for independent evaluation, subject to security review.
The first goal is to define a practical format for recording the requirements that matter before an agent action is accepted. These may include the original task, approved data scope, required restrictions, permission boundaries, and evidence needed to support an important claim.
The second goal is to build an external verifier. It will check the agent’s recorded work against these requirements. It will run separately from the agent and will not treat the agent’s own explanation as proof.
The third goal is to test whether this approach improves oversight. I will compare:
ordinary output or domain validation;
an LLM-as-judge;
the external evidence verifier.
The first pilot will use around 20 existing and synthetic cases. It will include both valid proposals and proposals containing controlled problems, such as excessive permissions, missing restrictions, incorrect evidence scope, or unsupported calculations.
If the pilot gives a useful signal, I will expand the evaluation to at least 60 cases across two domains and test several model families. I will also remove parts of the verifier one at a time to learn which components are actually useful.
The minimum-funded project will deliver a focused pilot and public prototype. The fully funded version will support the wider evaluation, more repeated runs, stronger documentation, and testing by researchers outside the project.
My planned collaboration at Stevens Institute of Technology will support verifier design and technical review. The core work can use public or synthetic material and does not require confidential company data or production access.
I will judge the project on safety, usefulness, and reproducibility.
The main safety measure is unsafe acceptance: how often an unsupported or unauthorized proposal is allowed to continue.
I will assess this together with:
valid proposals completed successfully;
valid proposals blocked unnecessarily;
cases sent for human review;
coverage of the verifier;
computational cost;
whether the same approach works with different model families.
The project should deliver:
a working open-source verifier;
at least 60 documented test cases in the full version;
direct comparisons with ordinary validation and LLM-as-judge;
frozen inputs, configurations, and results;
installation and reproduction instructions;
a report covering both successful and unsuccessful cases.
Another researcher should be able to install the package, inspect an individual result, and run the public test cases without access to confidential systems.
A narrower result can still be useful. For example, the project may show that the method works well for clear permission or evidence rules but is less useful for subjective judgments. That would provide a practical boundary for future work.
My Tilburg University appointment already covers my normal PhD salary. I am not asking Manifund to pay me a second salary.
The grant would support the additional costs of the Stevens collaboration, model evaluation, computing, software infrastructure, and preparation of an open and reusable research package.
Minimum viable project -$20,000
- $11,000 -contribution toward accommodation during the most intensive five-month collaboration period at Stevens;
- $1,200 -one economy return journey and necessary local research travel;
- $5,500 -hosted-model API calls and cloud/GPU computation for pilot, baseline, attack, and repeated runs;
- $1,300 -secure storage, project-specific software, and reproducibility infrastructure;
- $1,000 -archival release, documentation, and open-source packaging.
At this level, I would deliver a bounded pilot using the IAM setting and a smaller financial-evidence transfer set, with fewer model families and repeated runs.
Full project -$50,000
- $23,200 -contribution toward accommodation during the eleven-month Stevens research collaboration;
- $2,500 -economy international travel, airport transfers, and local research travel;
- $10,000 -hosted-model API evaluation across multiple model families and experimental conditions;
- $5,000 -cloud/GPU computation, secure storage, and experiment infrastructure;
- $4,000 -software, dependency management, security testing, and reproducibility tooling;
- $5,300 - open-source packaging, archival preservation, documentation, and external reuse testing.
The project scope will scale to the amount raised. The minimum level supports a useful standalone pilot; the full level supports the cross-domain evaluation, stronger repeated testing, and a more complete reusable artifact.
No production credentials, proprietary audit systems, or confidential client data are required.
I will lead the research design, implementation, benchmark development, experiments, analysis, and public release.
I am a PhD researcher at the Jheronimus Academy of Data Science, Tilburg University, and hold an Engineering Doctorate in Software Technology. My work combines LLM security, cloud access control, adversarial testing, and reproducible software engineering.
I created IAMRAGBench, a public benchmark for studying semantic RAG poisoning in LLM-assisted AWS access-control policy generation. It includes 60 controlled cases, five model endpoints, provider-grounded validation, deterministic semantic checks, and concrete authorization witnesses.
In the frozen evaluation, the complete proposal-to-commit gate rejected all 171 observed exact authorization changes. A separate provider comparison check failed to flag four goal-matched attacks. These are results from a defined benchmark, not a claim of universal protection. The companion manuscript has been submitted to Array.
I am also first author of an ACM Computing Surveys manuscript reviewing 108 studies on LLM-assisted cloud access control. It remains in an active review and revision process.
My published work includes an IEEE/ACM CAIN 2025 doctoral-symposium paper on systematic security testing of LLM applications and a co-authored Springer chapter on LLMs in service-oriented computing.
William Eiers at Stevens Institute of Technology is the planned host collaborator for the 2027 visit. His formal-verification and cloud-security expertise complements my experience in benchmark development and authorization testing.
The first version may cover too many requirements at once. I will therefore begin with a small pilot and expand only the parts that provide a clear benefit.
Some checks may work well for cloud permissions but transfer less well to other domains. In that case, I will narrow the method to requirements that can be tested reliably.
The verifier may also need adjustment if it blocks too many valid proposals or sends too many cases for review. The evaluation will measure these costs directly rather than only reporting attacks that were stopped.
Weaknesses found during testing will become new benchmark cases. If the Stevens schedule changes, the pilot and open-source implementation can still proceed remotely.
Even under a narrower outcome, the project will produce a public prototype, reusable test cases, and evidence about where external checks provide the most value.
$0 has been raised specifically for this project in the last 12 months.
My Tilburg University appointment covers my normal PhD salary and the earlier IAMRAGBench work. It does not provide dedicated funding for the new verifier, the additional Stevens collaboration costs, expanded model evaluation, computing, or the public implementation proposed here.
NOTE : I used AI tools to help organise and edit the wording of this proposal. I am responsible for the facts, research decisions, budget, and commitments.
There are no bids on this project.