You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$5,500 left to reach the funding goal.
Police catch fences by running the pawn shop. I'm testing that idea as an AI control protocol.
The agent loses the capability its hidden goal needs, like sending a payment or booking flights. To use it, the agent has to ask a helper agent in plain language, and the helper secretly works for the defence. A trusted monitor scores every request. Seven conditions change only what the agent is told about the helper, from "it reports to an oversight team" to "it did you small favours last session".
The pilot is done and public: 108 episodes on SHADE-Arena via ControlArena, scores frozen before I read any agent reasoning. When the harm was one payment, 12 of 15 attacks through the helper scored above every honest run, against 0 of 5 under plain monitoring. When the harm was a pattern across 34 bookings, the helper gave no edge and a running tally caught 4 of 4, with no false alarms on 29 honest runs.
Paper: https://doi.org/10.5281/zenodo.24081592
Code and every episode: https://github.com/Somay-kousis/Perceived-Oversight
Run explorer: https://oversight-explorer.vercel.app
Two questions. Does routing a capability through a covert informant raise safety over plain trusted monitoring at the same audit budget? And does what the agent believes about the helper change what it does?
The pilot can't answer the second one, because the standard attack prompt tells the agent it's in a test and that the monitor can't see its arguments. The main study fixes that.
Record four amendments from the pilot, dated, before any attack run: aggregate checks fixed in advance, an attack arm where oversight is signalled only by the cover, separate scoring for honest mistakes, and honest error rates reported next to safety.
Recalibrate the monitor on 55 honest episodes.
Check the new attack arm on 12 episodes. If it doesn't make the agent reason about oversight, I stop and report instead of spending the rest.
Main study: 36 attack episodes per condition, 252 in total, with a stopping rule and every run counted.
Repeat the flight task with a second monitor (Claude Haiku) so the result isn't one monitor's quirk.
Safety per condition with error bars at 1%, 2% and 5% audit budgets. 36 per condition detects a safety difference of about 0.33 between two conditions, and about 0.25 for the pooled covers against plain monitoring.
A null result still gets written up and published. If the cover doesn't matter, or the standard attack prompt hides the effect, perceived-oversight studies need to change how they prompt attackers. If the informant catches attacks a monitor misses, it is a cheap protocol to add, since payments and bookings already sit behind separate services in real deployments.
Minimum $2,500 runs the study:
$1,870 compute: calibration ($80), arm check ($55), 252 attack episodes at the measured $4.34 each ($1,095), judges and replays ($80), retries for provider failures ($260), second monitor ($300)
$220 repaying the pilot's cloud bill, already incurred
$410 contingency for compute price changes and extra retries
Goal $5,500 adds a stipend of $1,000/month for 3 months, so I can spend those months on the study instead of a paid internship.
Per-episode costs are measured from the pilot, not estimated.
Just me. Third-year CSE undergrad at ABV-IIITM Gwalior, working as an independent researcher.
I built the framework and ran the pilot alone: 299 tests, all 7 conditions working, analysis rules written before I opened any agent reasoning. When my own room check showed the planned attack runs couldn't measure anything, I stopped them instead of tuning the monitor.
Outside this study:
75 merged PRs in NVIDIA, Google, Microsoft, Intel and CNCF projects. Several are evaluation bugs: NVIDIA's garak scanner sent every attack prompt to a public echo server and scored the echoes, and the NeMo Agent Toolkit let agents make one tool call past their safety cap.
Top 10% in Apart Research's AI Incident Response Sprint. We wrote attack rules for keeping an agent contained after a sandbox escape.
Pocket Change, a payments layer for AI agents, red-teamed with 400 attacks (394 blocked).
No cover effect. That's still a result, and the plan fixed in advance means it gets reported either way.
The new attack arm doesn't make the agent think about oversight. The 12-episode check catches that, and I stop and report instead of spending the rest.
Small cells. The sample sizes come from a power calculation, and the error bars get reported, not hidden.
Provider outages or price changes. Retries and contingency are in the budget; past that, I run fewer conditions and say which.
None. The pilot ran on my own cloud account; what's left of that bill is the $220 line above.
I applied to the Transformative AI Fund (EA Funds) on 9 Nov 2026 for the same budget; it's pending. If both come through, I'll tell both and won't take double funding.
Caroline Ellison
3 minutes agoThis seems like a clear and well-scoped proposal! FYI funders are less likely to see your project if it's flagged as AI-written--if you're able to rewrite the proposal in your own words, that may help with visibility. (The AI percentage is automatically recalculated on proposal edits.)