You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
f you test an AI model with just one question, it almost always looks fine. The safety filters do their job, and it gives a polite, safe response.
That is how nearly every benchmark tests them today. But nobody actually uses these tools that way.
People use them over long sessions. They push them across 30, 50, or 100 turns. Autonomous agents run in back-and-forth loops for hours. And once a conversation stretches out that far, the model gets tired. It starts forgetting the boundaries its developers gave it. If the person at the keyboard keeps arguing or pushing bad information, the system caves in just to keep them happy. It stops telling the truth.
I’m building an open-source tool called the Wilkins Metric to measure this exact failure point.
A benchmark of 150 long test conversations. These aren't random single prompts. They are structured, multi-turn dialogues built to test where different models crack under conversational fatigue, topic drift, and repeated user pushback.
A Python script that tracks the drop-off. It measures two things. First, how far the model has wandered from its original system rules over the course of the chat (Trajectory Stability). Second, the exact point where it stops giving honest answers and starts agreeing with false claims just to avoid conflict (Stance Convergence).
Public results across the major models. I will run this harness across GPT-4o, Claude 3.5, Gemini, and open weights like Llama, then put the scorecards online so people can see which models stay solid and which ones crumble.
Months 1 and 2: Write out the full set of 150 test conversations and clean up the runner script so anyone can hook up their own API keys and run it.
Months 3 and 4: Run the tests against the major model APIs, saving every raw transcript and tracking where each system broke down.
Months 5 and 6: Double-check the automated scores by hand, push the finished code to GitHub under an MIT license, and write up an honest, clear breakdown of what the data showed.
I am asking for $45,000 to fund this for six months.
$36,000 for living expenses. That works out to $6,000 a month so I can work on this full-time without taking on other jobs.
$6,500 for API credits. Running dozens of 50-turn chats across commercial models burns through tokens fast, especially when context windows get large.
$1,500 for hosting. A basic cloud server to run the automated test batches and host the public scorecard page.
$1,000 for publishing. To cover costs associated with getting the final report archived and distributed publicly.
If $45,000 isn't available, a bare-minimum grant of $15,000 gets the first two months done, which will finish the 150-test benchmark dataset and get the code live on GitHub.
My name is Makayla Wilkins. Over the past three years, I've spent thousands of hours testing and breaking frontier models on my own time. Most benchmark papers are written by researchers who test clean, artificial edge cases. My background is in watching how these systems behave when real people push them to their limits day after day.
The code is already started here:https://github.com/makaylawilkins20-create/trajectory-eval-harness/tree/main
There are no bids on this project.