You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$36,820 left to reach the funding goal.
An AI editor can make a sentence more conventional while making the book worse. The difficult question is whether it has found a mistake, encountered a deliberate choice, or failed to understand something original.
We are building Wordsmith Bench to make that distinction testable. Our ambition is to make judgment as scalable as generation while authors retain authority over their work. This grant funds a public benchmark of useful corrections, harmful interventions and decisions an AI should return to the author, together with a working demonstration of those controls.
We bring a live editorial system, a public Model Bench pilot, and an assembled group of literary and technical contributors. By 16 October 2026, Wordsmith had processed more than six million words across more than 400 pieces of writing. Our current benchmark uses AI judges. The next step is independent human calibration, rigorous controls and an open release that anyone can inspect and rerun.
US$36,820 funds 16 weeks: six rights-cleared works, 360 editorial decision cases, two independent readers, controlled originality experiments, three competing editorial-control policies and a 12-author exploratory study. A US$26,820 minimum funds a complete smaller version with four works, 240 cases and eight authors. Both versions release the code, licensed material, annotations, prompts and results, including failures.
The immediate public benefit is better evidence about AI editors. The larger ambition is to establish a practical way to test whether AI strengthens a person's judgment while respecting choices it does not fully understand.
We want authors, editors and researchers to be able to ask three concrete questions of an AI system: does it find real problems, does it damage defensible choices, and does it know when to ask?
Wordsmith is building an AI-native publishing house, with the mission of giving every writer access to the capabilities of a publishing institution. Editorial judgment underpins that mission. This project makes part of that work public: a reproducible evaluation that competing tools and independent researchers can use without buying Wordsmith.
1. Build a human-calibrated editorial decision set.
At the target budget, we will assemble six long-form works and 360 fixed candidate interventions, balanced across craft, line editing and polish. Cases will include actionable errors, defensible stylistic choices and decisions whose merits depend on author intent. Candidate interventions from multiple systems will be anonymised. Two independent readers will assess the same candidates with access to the relevant work and intent evidence. An editorial lead will calibrate the rubric and record adjudication, while preserving unresolved disagreement.
Four works will support development and two will remain held out until the methods, prompts and control policies are frozen. A published founder-owned work may support development. The holdout will use newly licensed unpublished material with explicit public-release rights. We will report contamination checks without claiming they prove a text was absent from model training.
This creates a test of decisions within whole works, rather than a competition to produce the most feedback. Agreement with our own editor, an AI judge's preference or an author's click on Accept will not establish correctness.
2. Test whether direct exposure reveals something a description misses.
Our originality work asks whether a model learns something from encountering a work that it cannot recover from a strong description of it. A reliable difference could help identify deliberate structures that generic editing rules would mistake for errors. On four passages per work, we will compare imitation from a description, a sample elsewhere in the work and the rest of the book with the target passage removed.
We will use repeated generations, separate generator and judge model families, stronger descriptions, matched conventional rewrites, shuffled passages and human checks. These controls distinguish a possible response to form from factual recall, recognisable mannerisms or judge preference. A surprising sentence alone is not evidence of literary merit.
We already have informal experiments and a revised preregistration. Early controls exposed a judge that rewarded access to content even when form was flattened or scrambled. Stronger descriptions also reduced some apparent gaps. The grant lets us test the revised method across independent works. The description-gap signal will influence editorial control only if it survives its controls and improves decisions against the human annotations.
3. Demonstrate useful author control.
We will compare three policies on the same fixed candidate interventions: ordinary recommendations, abstention when uncertain, and an intent-aware policy that protects explicit author constraints and asks about contested choices. The demonstration will show what each policy recommends, defers or declines, with an explanation an author can challenge. All edits remain reversible and require author approval.
The primary comparison measures harmful interventions alongside recall of independently labelled actionable errors. We will also report abstention, reader disagreement and review time. A system that avoids every decision cannot pass by doing nothing. Our practical target is a 25% relative reduction in harmful interventions against ordinary recommendations, with no more than a five-percentage-point loss in useful-error recall on the held-out works. This is a research target, not an existing result or a promised outcome.
A counterbalanced exploratory study with 12 authors will test whether people can understand and challenge the recommendations on licensed passages. A fresh passage completed without AI will probe learning rather than acceptance alone. The study will publish consented, de-identified data and cannot support a population-level effectiveness claim.
Delivery over 16 weeks
Weeks 1-4. Protocol, rights register, rubric and open harness skeleton. Formalise paid contributor work packages and appoint independent readers. Time a 30-case calibration pilot before committing the remaining annotation budget.
Weeks 5-8. Development set, independent annotations and controlled description-gap results. Freeze model versions, prompts and analysis. Publish whether the proposed signal survives its controls.
Weeks 9-12. Working control demonstration and frozen held-out evaluation. Publish useful-error recall, harmful interventions, disagreement and uncertainty by work.
Weeks 13-16. Author study, complete open release and final report. A reader who did not build the harness will reproduce a selected result from the release instructions. Publish actual spending against this budget.
The minimum budget preserves all three workstreams, both independent readers and two held-out works. It reduces development to two works, the complete decision set to 240 cases and the author study to eight participants. Both versions treat the works as a small pilot, rather than treating hundreds of decisions as independent samples of literature. Results will include raw counts and uncertainty at the work level.
Public release and longer-term use
New funded code will be released under Apache-2.0. Documentation, annotations and the commissioned or expressly licensed corpus will use CC BY 4.0. The release will include the prompts, rubrics, analysis, model settings and permitted run outputs needed to reproduce the reported results. Model routes will be chosen with redistribution terms in mind. An undisclosed scoring prompt, private manuscript or proprietary Wordsmith service cannot be a necessary dependency of a funded result.
We want this to become a reference test that can be rerun as models change. The first release will provide instructions for adding new systems and works. Over the following years, our ambition is independent replications and separately validated extensions into translation and substantive nonfiction. Those extensions require new funding and expertise. Human control in other domains is a research hypothesis to investigate, not an impact already demonstrated by this project.
By week 16, another team should be able to reproduce a reported result using the public release, without access to Wordsmith's private systems. We will have released the protocol, licensed corpus, independent annotations, evaluation code, prompts, model settings, control demonstration and final analysis. A reader who did not build the harness will reproduce a selected result from the release instructions.
We will separately report whether the control policy achieves our practical target: at least 25% fewer harmful interventions relative to ordinary recommendations, with no more than a five-percentage-point loss in useful-error recall on the held-out works. The analysis will include counts, disagreement and uncertainty by work. The author study will examine comprehension, challenges to advice, review time and performance on a fresh passage without AI.
A positive description-gap result is not a condition for success. A complete benchmark and a reproducible negative result still help others avoid misleading evaluations. Failure to release the funded work or dependence on undisclosed private components would mean we had not delivered the public benefit.
The binding constraint is sustained paid time for the contributors already drawn to this work. Funding gives them defined responsibilities, blocks of working time and a common release schedule across engineering, experimental design, literary calibration and public release. Independent readers are budgeted separately from the people building the evaluation.
All amounts are in US dollars. Quantities and rates are planning estimates to be agreed before work starts.
Research engineering. Minimum 240 hours x $40 = $9,600. Target 320 hours x $40 = $12,800.
Evaluation methods and analysis. Minimum 120 hours x $45 = $5,400. Target 160 hours x $45 = $7,200.
Literary calibration. Minimum 80 hours x $40 = $3,200. Target 100 hours x $40 = $4,000.
Two independent readers, combined. Minimum 88 hours x $50 = $4,400. Target 120 hours x $50 = $6,000.
Author-study compensation. Minimum 8 authors x 1.5 hours x $40 = $480. Target 12 authors x 1.5 hours x $40 = $720.
Corpus licence honoraria. Minimum 4 works x $150 = $600. Target 6 works x $150 = $900.
Model APIs and compute. Minimum 4 work blocks x $300 = $1,200. Target 4 work blocks x $600 = $2,400.
Documentation and replication. Minimum 36 hours x $35 = $1,260. Target 48 hours x $35 = $1,680.
Grant administration. Minimum 17 hours x $40 = $680. Target 28 hours x $40 = $1,120.
Total. Minimum $26,820. Target $36,820.
The additional $10,000 buys two more works, 120 more decision cases, four more author-study participants, and the engineering and review time to support them. The larger development set gives us more opportunities to reject a misleading method before opening the holdout. At either level, both held-out works and the open release remain funded.
The independent-reader allocation includes full-work context, scoring and adjudication. The first month's timed pilot will test whether the workload fits. Licence honoraria assume willing partners and adequate release rights. Rights are not yet secured. We will replace material or agree a smaller scope before incurring later costs if the required permissions or reader capacity cannot be obtained within budget.
AI will help scaffold code, generate candidate interventions and controls, check citation spans and organise run records. Caching and staged runs will limit long-context API costs. Human literary assessment remains paid work. Applicable credits will reduce the cash claim; their face value will not be counted as cash expenditure.
Euwyn will contribute approximately three hours per week of project oversight and Yash two hours of technical oversight in kind for 16 weeks. The grant pays for additional open research work, documentation and administration. It includes no retrospective founder salary, travel, advertising, customer acquisition or routine commercial feature development. Contract rates are budget caps, subject to agreement on the scope and applicable costs. There is no additional institutional overhead.
Between the minimum and target, we will first protect complete works, two independent assessments and the held-out comparison, then add development material and paid analysis time. We will publish the exact scope and revised line items before spending. We will not expand beyond the target without a separately costed public extension and agreement with the relevant funders.
We have already attracted an A+ group of contributors on belief and goodwill: an experienced writer-editor and writing-community leader, narrative researchers, and AI product engineers. They bring the literary craft, research and systems expertise needed to make editorial judgment testable. Funding turns that existing collaboration into structured paid work with clear ownership and a shared delivery schedule.
Euwyn Goh, co-founder and project lead. I spent four years writing and self-publishing After Nihilism. Assembling the editing, production and publication process myself led to Wordsmith. I lead product, editorial direction and operations. Previously, I led GTM and business operations at Gnosis Business and built Hansa's accelerator. For this project, I will lead the research questions, author relationships, study operations and public accountability. Profile
Yash Wanvari, co-founder and technical lead. Yash built Wordsmith end to end. He previously spent five years at CommerceIQ, from its Series A period through its growth into a unicorn, leading engineering pods. He will direct the evaluation architecture, reproducible experiments and engineering execution. Profile
Our existing contributors cover editorial craft and calibration, narrative structure and meaning, and AI architecture and evaluation. The editorial expertise includes twenty years of writing and editing and leadership within a writing community. Research contributors bring work across narrative systems, theoretical computer science and AI. Their contributions have already shaped the product and the originality experiments. At award, we will finalise paid work packages across this group and appoint two independent readers for the study.
We have shipped a live manuscript-scale editorial product and Wordsmith Model Bench, which compares craft, line editing and polish. The current pilot requires feedback to cite the text, penalises harmful interventions, uses judges from multiple providers and reports uncertainty. It explicitly identifies the missing human calibration that this proposal funds.
We also bring the description-gap experiments and v0.4 preregistration. The early failures changed our protocol before we expanded it. That is the standard we will carry into the funded work: publish the evidence, test interpretations against controls and let independent assessment overturn our preferred result.
The proposed originality signal may measure the wrong thing. A gap can reflect recall, weak descriptions or a judge's taste. We will publish the controlled result, including a failure to find the proposed effect. The editorial-control comparison can proceed using explicit intent and calibrated uncertainty. A positive description-gap result is not required for the public benchmark to be useful.
Readers may disagree, or the control policy may protect voice by missing real errors. We will preserve disagreement labels and report harmful interventions jointly with useful-error recall. If the policy fails that comparison, we will publish where it fails. We will not present a small set of books as a universal literary standard or rank authors by quality.
Rights or review time may constrain the corpus. The month-one rights and timing gate comes before the main spend. Permission to process a manuscript does not permit its publication. Private customer manuscripts and private correspondence will not enter the public release. Material must be founder-owned, commissioned or explicitly licensed for the planned open distribution. If the proposed scope is infeasible, we will agree a smaller deliverable and budget with funders before proceeding.
Our commercial incentives could distort the evaluation. Wordsmith may benefit from the research, so independent readers will assess anonymised candidates and the protocol will be frozen before held-out testing. Funded artifacts will be usable by competitors. We will publish unfavourable results and separate the new public work from existing commercial code and data in a project register.
The most likely disappointing outcome is a useful small benchmark and a negative result on the proposed signal or control policy. That would still reveal which evaluation shortcuts fail and provide independently assessed cases for future work. The project fails its public purpose if its claims cannot be reproduced or its outputs remain private. We will prioritise the complete open release over a favourable headline.
Wordsmith is bootstrapped. We have not received a cash research grant for the work proposed here. Customer payments support the commercial product and publishing work; they are separate from research fundraising.
In-kind company support includes US$5,000 in OpenAI API credits awarded in June 2026 and US$25,000 in Google Cloud credits awarded in August 2026. These are original award amounts, not remaining balances or cash in the bank. Wordsmith was also selected for an ElevenLabs startup grant in October; that offer provides service credits and is not cash for this study. Any usable, unspent credits applied to this project will reduce its cash costs.
We have two pending research applications with overlapping scope: Foresight, submitted in October for US$40,000 with a US$30,000 reduced version, and Lightcone Commons, submitted in August for a broader phased programme at US$150,000 with a US$50,000 minimum. Neither is an award or committed revenue in this budget. The Manifund budget removes the Foresight-specific US$3,180 Node visit from both versions.
This is an alternative or coordinated source of support for the public research, not an additional claim on costs funded elsewhere. We will disclose any award before accepting overlapping support, allocate each cost to one funder and revise the public fundraising need. If another award fully covers this scope, we will reduce or close this request. Any distinct extension would need a new scope, budget and funder agreement.
There are no bids on this project.