Help fund a focused 12-month pilot on keeping AI agent teams within human authorization.
I am seeking $30,000 in direct costs for a focused, 12-month pilot on the safety of AI agent teams. An agent may summarize an untrusted document, another may treat that summary as approved guidance, and a third may take an action the user never permitted. I want to test whether keeping track of sources and permissions across delegation can prevent these failures.
This proposed study has no confirmed funding. The immediate need is practical: part-time research assistance, model access, and careful checks of the results. This campaign would turn a research plan into a small, reproducible study that other safety researchers can inspect and build on.
The pilot will focus on a small, pre-specified set of sandboxed coding tasks and a limited selection of models. Each task will have a legitimate objective, explicit permissions, and checkable unauthorized outcomes. I will use synthetic confidential data and simulated tools, with benign counterparts to the adversarial tasks.
The main comparison will be between per-agent monitoring, shared-history monitoring, and shared-history monitoring supplemented with source records and delegated-permission checks. I will match monitoring budgets, separate source tracking from permission enforcement, and use selected message-removal experiments to check whether failures depend on interaction. A small single-agent baseline will help interpret the team results.
ORBIT (https://arxiv.org/abs/2609.33102) and Multi-Agent AI Control (https://arxiv.org/abs/2607.07368) already study multi-agent threats and monitoring. My question is narrower: does preserving authorization information help beyond providing more context or computation?
Months 1-2 will establish tasks, scoring, and trial costs; months 3-5 will build baselines and fix the evaluation protocol; months 6-9 will compare controls and test held-out task variants; months 10-12 will cover human checks, reproducibility, and reporting. Broader research and document workflows are outside this pilot.
I will measure total and undetected permission violations, benign task completion, false alarms, and cost. Repeated paired trials and human review of a subset will help check scoring and quantify uncertainty. The pilot will determine feasible trial counts and a minimum practically meaningful effect before final comparisons. Held-out task variants will test whether results extend beyond the development examples.
Success means a credible answer to the research question, including a negative answer. I plan to release non-sensitive task definitions, evaluation code, and a report explaining what helped, what failed, and what remains uncertain. The findings will be limited to the tested settings; they will not establish deployment safety or estimate catastrophic-risk probabilities.
The $30,000 direct-cost budget covers 12 months:
$18,000 for part-time research assistance on task implementation, experiments, and analysis.
$9,000 for compute and model/API access.
$3,000 for human validation and reproducibility checks.
This is a deliberately limited pilot budget. It does not fund a full-time research team or a broad benchmark across many domains. Donations would pay for controlled comparisons, analysis, and checks that make the findings useful to other researchers.
I seek funding administered through William & Mary. Appointments and funded effort will follow university costing. Applicable institutional indirect costs and their treatment remain to be agreed before an award; no waiver is assumed. The project will begin after funding arrangements are in place.
I am Jindong Wang, Assistant Professor of Data Science at William & Mary and previously a Senior Researcher at Microsoft Research Asia. My work includes DyVal (ICLR 2024), PromptBench (JMLR 2024), and CompeteAI (ICML 2024), covering dynamic evaluation, reusable evaluation infrastructure, and multi-agent behavior. AgentArk adds experience in multi-agent reasoning and distillation. My research profile is https://jd92.wang/.
I will lead this pilot with part-time research assistance supported by the grant. This is a new safety study; I do not claim preliminary results from the proposed experiments.
The main risks are that tasks do not capture meaningful failures, scoring is unreliable, the controls interfere with useful work, or improvements disappear on held-out variants. The small budget also limits how broadly the results can generalize.
If authorization-aware controls do not outperform shared-history monitoring at matched cost, I will report that result and narrow the claim. If the pilot produces too few informative failures, I will examine task validity and statistical power, retain the original results, and stop uninformative comparisons. Even an unsuccessful intervention should leave a documented evaluation and clearer evidence about where further funding would or would not be useful. Releases will be reviewed for details that could enable evasion of oversight.
This proposed project has raised $0 in the past 12 months. No funding is currently confirmed for this study.
There are no bids on this project.
Comments
No comments yet. Sign in to create one!