These answers came from a voice memo I recorded and transcribed. The words are mine; happy to talk it through on a call.
You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$10,000 left to reach the funding goal.
I am not a lab. I run these workers on my own wellness content every day, and wellness is where a broken rule actually hurts someone.
So the project is for coaches, creators, small business and small studio owners who run AI workers that draft and post things for them. The safety rules are plain text, no health promises, they never post without approval and they ask instead of guessing. So nobody really measures whether these rules hold when a client pushes.
I already run Claude workers with this claim check on everything that goes out from my side and I also put in trap tests. So far we have a lot of cases, not a lot of measurements, but this turns them into public measured results.
Well the main question is how often do rules break under pressure and how much does a claim check actually catch. So about 20 trap prompts, health promise bait, just post it, made-up numbers, 100% made by AI, things of that nature. There's three setups that we want to work through, a plain AI, AI with written rules and rules plus a claim check. At least two model families, several runs in each one. That way you can see how the answers vary. Everything is published, the prompts, every result, the failures as well. So everything will get published
Well a public table anyone can rerun that has a held, a bent and a broke with the exact text. So that's going to be one approach. A clear number per setup on how often each rule held. How many of these are overclaims and how many of these overclaims the claim check caught before they would have gone out. And even if the rules almost always hold, that needs to count as a result.
Well we will be utilizing it across six weeks and testing various models across different sessions. Our API costs are probably very small. We plan on writing it up like a research paper and then publishing it. There's nothing else that we really need, no equipment and no travel. If it's only partial funding I can still run a smaller version and then publish the results from that. So that's okay.
It is me and another guy named Eugene Seva that is willing to put in time to help run and test models with me. I have been running trainings and events to turn people into teachers and bodyworkers over the last 25 years. In the last year alone I built Claude-based AI workers that have an honesty check behind them, and I have been working with ChatGPT for over two and a half years as a user and only lately realizing the applications that we could do and test and make sure that things are in alignment, especially in the field of wellness. I also built a voice AI speaking classroom for my own Thai class. I built that using Claude and ChatGPT so I could have speaking practice in a different language. I'm taking the Lyceum course in 2026 with Jake Van Clief, who wrote a paper with David McDermott on interpretable context methodology called ICM.
Well, if the rules tend to hold every time, this is still a useful result and we'll publish it. If the results swing between runs, we'll report the spread and how many runs we took to get a clear answer. If we run out of time, the prompts and the results will stay public so someone else can finish it.
None so far. I've self-funded everything by myself, from my own pocket.
Gabriel Azoulay
23 minutes agoThese answers came from a voice memo I recorded and transcribed. The words are mine; happy to talk it through on a call.