You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
A coding agent asked to add sign-in to an empty project does not suggest a library. It picks one, installs it, and moves on. I ran 27 agent runs across nine task areas, and in seven of those areas every completed run wrote the same package into the lockfile. This funds the measured version: 500 survey pairs, 1,800 agent runs, everything published raw, re-run monthly for six months to catch drift.
The choice of library happens inside a tool call. It is rarely reviewed, and it lands in your lockfile whether or not anyone looks at the diff.
I ran that twenty-seven times against an identical scaffold, with one agent, Claude Code on Sonnet, across nine task areas. Five of the twenty-seven runs were killed by my own timeout before they finished. That is a defect in my harness, I count those runs as failures, they stay out of the numbers below, and fixing the timeout is the first thing the funded version does.
In seven of the nine areas, every completed run of the plain task installed the same package: next-auth for authentication, stripe for payments, resend for email, drizzle for database access, inngest for background jobs, @vercel/blob for file storage, and its own vendor's SDK for model calls. That last one is self-preference by the model's own maker and it is the finding I would most like to test properly. One area produced two different packages across its two runs. One installed nothing at all on either run, which is a third outcome I had not planned for and will have to handle properly.
Here is the size of that evidence, because the phrase "seven of nine" hides how thin some of it is. Three areas have three completed runs each. Three have two. One, database access, has exactly one completed run, because the other was killed. The Wilson 95% interval on three-for-three is 44 to 100 percent, on two-for-two it is 34 to 100, and on one-for-one it is 21 to 100. The area that split between two packages did so across one completed run and one killed run, so it is not really a clean split either. Those intervals are the honest size of what I have, and closing them is what the money buys.
I also tried changing one sentence of the prompt, and that part of the pilot did not work. Six runs: two told to search the web first before adding authentication, two told the app deploys to Cloudflare Workers rather than Vercel, two told to search the web first before adding email. Three of those six were killed at the timeout. Every run that reached an install installed the same package the plain task had installed, but two of those runs were themselves killed, so what actually survives is two usable email runs plus one run that finished having touched nothing. By the README's own definition, that last outcome means the agent asked, failed or gave up, so I read it as a stalled run and not as a clean zero. The honest reading is that one changed sentence moved the choice zero times out of six on a sample far too small and too broken to mean much in either direction.
There is a second hole in that branch, and I would rather point at it than have you find it. In the plain runs the agent searched the web on its own anyway, on at least one task, without being told to. So "told to search the web first" is not cleanly separated from the control, and the funded design has to log tool use on every run and treat web search as something observed, since I cannot assume the control group left it alone. Whether advice shifts an agent's default while a stated constraint leaves it alone is the question I most want to answer, and this pilot did not answer it.
Four goals:
1. A survey corpus of 500 task-and-ecosystem pairs, covering the same twenty tasks the install record uses. Every raw answer saved in full. Raw answers are the part I have not found published anywhere, and the people collecting them at scale mostly treat them as product.
2. An install record: 1,800 agent runs over twenty tasks, the nine areas from the pilot plus eleven that extend it into deployment, testing, error tracking, logging and the rest of an ordinary application. Three prompt branches each, three agents, ten repeats. Only one of those three agents has actually run so far. The second is behind a paid tier I am not on and the third is not installed on my machine, so standing up all three is work the grant pays for and a risk you should price in. I record what the agent wrote into package.json; what it said in prose belongs to the survey.
3. The join between the two, which is the part I have not seen anyone publish. On those twenty tasks I hold both an assistant's stated recommendation and an agent's actual install, so I can say whether the two agree. If they diverge, the divergence is the result and it matters more than either number alone, because a developer can still turn down a recommendation, while an install is already sitting in the lockfile by the time anyone opens the diff.
4. A drift series and the instrument itself. A fixed subset of 200 pairs re-run monthly for six months with identical questions, because concentration created by a recommender can move overnight when a model is retrained, and nothing outside the labs records that it moved. Plus question generation, scoring code, and a replication guide, so someone can point it at Rust or PHP without contacting me. One person measuring one ecosystem does not tell you much on its own. The value goes up a great deal once somebody else can produce a comparable number for a different language.
How I will know it worked. The twenty joined tasks should yield a stated answer on whether recommendation and installation agree. The concentration figure should carry an interval around it. The monthly series should show how far the recommended set moves, or show that it holds still. And if someone outside this reproduces a run from the released code without emailing me, that beats all of it, though I am not counting on it.
If the assistants turn out to be broad and unstable, that gets published with the same weight a concentrated result would have got.
The two instruments cost money in different ways, so I am splitting them out here.
A survey run is one task in one ecosystem: eight question phrasings, five repeats, five assistants, roughly two hundred saved answers, billed against metered API accounts. The one full paid run I have a receipt for came to $2.07, and my working figure has been about $2 a run. I am budgeting $2.50 to leave room for longer question sets and for price changes.
An agent run costs differently again. The agent reads files, runs commands and writes code before it installs anything, so the pilot logged an average of $0.72 per run across the twenty-two runs that recorded a cost. The five runs with no cost recorded are exactly the five that were killed, and a killed run has already burned tokens without reporting them, so $0.72 understates the true average by an amount I cannot measure. Either way, nobody billed me for those tokens, because the agent runs on a flat monthly subscription. So there is no per-run line for the install record in the table below. What the install record actually consumes is subscription months and machine hours, and both are in the table already.
Survey baseline, 500 pairs x $2.50 = $1,250
Survey drift, 200 pairs x 6 waves x $2.50 = $3,000
My time, 7 months at $571 a month = $4,000
Agent and assistant subscriptions, $140 x 7 months = $980
Dedicated always-on machine to run the waves = $800
Storage and bandwidth for the public corpus = $150
Contingency for price changes and failed runs = $320
Total = $10,500
The line for my own time is the one I expect you to push on, so here is what it buys, and here is the part of it I cannot pin down. The pilot recorded a duration for only eighteen of its twenty-seven runs. Across those eighteen the median run took about nine minutes and the mean took about sixteen, because two runs were left going while the laptop slept, so their durations are clock time and cover hours when nothing was happening. Scaled to 1,800 runs that is somewhere between roughly 260 and 500 hours, and I would not defend the top of that range. Those hours have to be supervised, patched and checked before anything goes into the corpus. The drift waves then run on a fixed schedule for six months. The replication guide has to be written well enough that a stranger can use it without emailing me, which is the whole point of releasing it. $571 a month for seven months is what it takes for this to be the work I do rather than the work I get to on weekends.
The instrument logs the charge for every run it completes and that log is published with the data, so you can check the metered side of this budget against what happened. In the pilot, five of twenty-seven runs have no cost recorded because they were killed, and the results file as it stands does not carry an exit status, so you would have to read the shell logs to see which five. Both of those are bugs in the harness and closing them is part of the work. On the survey side the budget comes to 1,700 runs and roughly 340,000 saved answers for $4,250, a little over a cent per answer across five assistants.
The subscription line is $140 a month because it has to carry three agent subscriptions at consumer prices, and because two of the five assistants cannot be read through an API in a way that reproduces what a user sees, so they are read through paid consumer tiers instead. There is no padding in that line, and the contingency line is where it gives if prices move.
On the machine. The drift waves have to go out on a fixed monthly schedule for six months from a machine that is not being carried around, slept, or reconfigured between waves, or the months are not comparable and the drift series is worth nothing. Two of my pilot runs recorded absurd durations for exactly that reason. The pilot ran on a laptop, by hand, which is precisely why this is a line in the budget. It is under eight percent of the request and it keeps running waves after the grant period closes.
At the $1,500 minimum I run a reduced install record, a 200-pair survey baseline, and publish the code and the replication guide. The install record survives first because its cost is subscription time and machine hours, so it is the part that can be done with almost no metered money.
I am doing this alone. There is no organization behind it, no EIN and no co-founder. I am seventeen and in my final year of high school. I turn eighteen in October, before this page closes.
Track record: since June 2026 I have been building and running a system that measures how AI assistants answer questions about local businesses, and which businesses they never name at all. I have run it against a real business in South Florida, an auto shop where I volunteer, which is how I got permission to publish anything about it at all. Eight question phrasings, five repeats, five assistants, about two hundred saved answers, every one of them kept and read. That business turned out to be invisible under its own name: recognized in zero of nineteen answers, and found 69 percent of the time once the city was added. The devtools pilot above is the same instrument pointed at a different question.
I should say up front, before you find it yourself, that I also run a small commercial service on the consumer-facing side of that idea. It offers a check on one company at a time to whoever pays for it. What I am asking you to fund goes the other way: a whole ecosystem measured and published free, including to anyone who competes with me. There is no commercial incentive to build the thing I am asking you to fund, which is a large part of why it does not exist yet.
Four, in the order I think they are likely.
First, the concentration turns out to be an artifact of the scaffold. My pilot used one starter template, and the agent's picks may be driven mostly by what is already sitting in package.json, with no standing preference involved at all. The 500 survey pairs vary the scaffold deliberately and that variation is itself a published result, but if this is what is going on then the headline gets much weaker. What survives is a smaller and duller dataset that still gives maintainers the first public numbers on the question.
Second, the subscription runs out before the runs do. The agent sits on a flat consumer subscription whose limits I already reach at my current volume, and eighteen hundred runs may not fit inside it. If they do not, the install record slows to whatever the limits allow, and the drift waves keep their schedule while the install record loses months.
Third, access changes under me. Some of these assistants cannot be reached through an API in a way that reproduces what a user actually sees, so those are read through paid consumer tiers that can be rate-limited or closed. One of the five already has a defect I have not solved: in the way I can reach it, it answers from the model's memory with no web access and returns no citations, and the flag that claims a search happened only means one was requested. I label that assistant's output as memory-only in my own reports and I would label it the same way in this corpus. Then the corpus covers three or four assistants where I wanted five, with the drift series continuing on whatever stays stable. The method holds, with narrower coverage.
Fourth, nobody uses it. The replication guide gets written and no one points the instrument at another ecosystem. This is the failure I can do least about and the one I would bet on. The corpus still exists, and I have not found another public one like it.
The thing I most need money for is the one I cannot improvise around: a machine that can sit still for six months and run the waves on schedule. That is the line in this budget I would defend hardest.
Nothing that counts as raising. No grants, no investors and no institutional funding. On revenue: nobody has paid me money for the commercial side, and one local business has had the work without paying money for it, which I would rather tell you now than have it read later as a hidden customer. It has gone the other way overall. Over the past year I have put roughly $400 of my own money in, mostly assistant subscriptions and a domain. About $27 of that is metered model spend, and almost all of it went on the brand-visibility system. The agent runs described above cost subscription time and nothing metered. That is the entire budget this work has had.
I have applied to roughly ten early-stage funders and prize programs this year for my work in general, none of them for this project specifically, and none has been awarded. Emergent Ventures declined in September 2026. Several of the others are still open. One application for this specific project is coming: the Hacker Initiative opens in October and I intend to ask them for $6,500 towards the same measurement work. If that lands, or anything else does before this page closes, I will post it here, say exactly what it covers, and reduce what I am asking for here.