You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$150,000 left to reach the funding goal.
An autonomous agent I run wrote a backtest result into its own long-term memory. No experiment by that name was ever run, the numbers matched nothing in its database, and it began recommending a live trading setup based on that claim.
I built claimcheck to catch this. It pulls numeric claims out of prose and checks each one against the structured data the agent really produced. It is MIT licensed, has no dependencies and has 49 tests. Code: https://github.com/meenadchisundaralinkamscooby-hash/claimcheck
With this funding I would test it on real agent transcripts, publish how often agents invent results, and build a guard that stops unsupported claims being saved to agent memory.
1. Measure the problem. Months 1 to 4: collect public agent transcripts that come with the tool output or data behind them, and label the numeric claims as supported or unsupported, with paid contract labellers so the labels are not only mine.
2. Publish it. Months 5 to 8: release the labelled set as a public benchmark, with the error rates I find and how well claimcheck does on it.
3. Ship the guard. Months 9 to 12: release a memory guard that flags unsupported claims before an agent saves them, plus a written report and documentation.
All work is public from day one.
The benchmark is public and other people can run it.
There is a published, reproducible number for how often agents state numeric results their own data does not support.
claimcheck has measured precision and recall on that benchmark, including the cases it misses.
The memory guard is released and at least one agent project other than mine has tried it.
Full goal, US$150,000, 12 months full-time: my stipend 96,000 (including tax), contract labellers 24,000, compute and API costs 15,000, buffer 15,000.
At US$15,000: 6 months part-time. A smaller benchmark labelled by me, the error rates, and the memory guard.
At the minimum, US$7,500: about 3 months part-time. A small labelled set and a first write-up of the error rates.
Just me. I live in Ontario, Canada. I am self-taught, with no degree, no paid developer job so far and no safety research background. I build with AI coding tools.
claimcheck exists and works on structured sources, but it is one commit with 0 stars and no users yet. I have no track record on a research project like this, so please weigh that.
This application was drafted with help from an AI assistant.
Public transcripts that include the data behind them may be hard to find, so the benchmark ends up small.
Labelling may take longer than planned.
claimcheck only checks numeric claims. If most invented results are not numeric, it catches a small share of the problem.
Nobody may adopt the guard.
In those cases there is still a public labelled set and an honest report on what did not work. Also, this reduces everyday agent errors more directly than it reduces catastrophic risk, so it may simply be a weak fit for safety funders.
US$0.
Pending, no decisions yet: the goose grant program (US$100,000, applied 2026-11-05, overlapping work: a goose extension that checks numeric claims against tool output), the EA Funds Transformative AI Fund (US$150,000, applied 2026-11-11, this same project), and a small Awesome Foundation Toronto application for an unrelated project.
If any of these is funded I will say so here right away and lower this ask.
There are no bids on this project.