If a model "passed at 85%", can you show that the 85% was picked before the run?

Usually not. The threshold, the metric, the dataset, the seed — they are all written down. The problem is they are written down in the same document as the result, and that document says nothing about which came first. Most of the time nobody cheated. The evidence just isn't there either way.

Medicine ran into this and answered it with pre-registration. AI evaluation is now starting to ask for the same discipline, but it still has no widely adopted, independently verifiable implementation layer.

PRML is my attempt at that missing layer, and it is small on purpose. Before the run, nine fields go into a canonical YAML file. The file is hashed with SHA-256. The hash gets an RFC 3161 timestamp from an independent authority, and can also go into the Rekor transparency log. Afterwards anyone can check the receipt on their own machine, offline. The verifier answers PASS, FAIL or TAMPERED, with exit codes you can drop into CI.

It does not prove the result is correct. It proves the bar was locked first. That is the only claim I make for it, and I would rather say it plainly than oversell it.

The code already exists. Four implementations — Python, JavaScript, Go, Rust — that produce identical bytes, with more than 600 tests between them. A public registry. Adapters for MLflow, Giskard, DeepEval, Hugging Face lighteval and the UK AI Security Institute's Inspect framework; the Inspect one is listed in AISI's extensions catalogue. All of it is open: falsify.dev, spec.falsify.dev, registry.falsify.dev, github.com/studio-11-co/falsify.

Why this matters for safety: recent work on ML agents found evaluator-tampering attempts in roughly half the episodes studied, and showed that locking the evaluator removed that attack surface (arXiv:2603.11337). That is runtime. My question sits one step earlier. Safety cases, dangerous-capability evaluations and RSP-style commitments all rest on criteria that someone set at some point. Right now you have to take the timing on trust. I want it checkable.

What are this project's goals? How will you achieve them?

Four pieces of work over the next six months.

The big one is execution linkage, $13,000. PRML can prove a manifest existed at a point in time. What it cannot yet do is tie that commitment to evidence from the run that resolved it, in a standard way, consistently across four implementations and the registry. Execution linkage is the main remaining technical gap.

Then v1.0, $9,000. Before freezing the spec I want a full cross-language conformance pass, so Python, JavaScript, Go and Rust agree on the same bytes and the same verdicts. Freezing a spec that four implementations disagree about would be worse than not freezing it.

Then an independent security review of the commitment path, $5,000. This is the smallest useful standalone piece here.

Last, key rotation and archival verification, $3,000. Boring but load-bearing: a receipt issued in 2026 should still verify offline in 2036, after the services and signing keys have moved on.

The full six-month plan costs $30,000. I am asking Manifund for up to $25,000; Falsify OÜ defers or covers the remainder. If this page is only partly funded, the first $5,000 goes to the independent security review.

I have also set a hard external test, and it is deliberately not a test I control. By 1 February 2027, either one relying party makes a PRML receipt a precondition for accepting an evaluation, or at least one paid engagement using PRML closes. A host framework embedding PRML as an opt-in feature counts too. If none of that happens, the project goes into maintenance mode instead of absorbing more money. 1 December 2026 is an interim check: I take stock of the signals, but no decision is made that day.

Correction, 29 Aug. This page previously gave 1 December 2026 and host-framework adoption as the hard test. The current gate is 1 February 2027, reflecting the timing of pending funding decisions, and the criterion now includes demand-side evidence: a relying-party requirement or a paid PRML engagement. Both changes were already in the internal decision record; this page had not caught up. On a project about fixing criteria in advance, changes to the project's own criteria should be explicit rather than silently edited.

That test matters to me because the whole point of this thing is to make evidence harder to massage after the fact. Earlier this year stricter validation rules made the registry demote its own flagship record. I left the failed record publicly visible instead of rewriting the history. That is the standard I am asking others to accept, so I have to live under it.

How much money have you raised in the last 12 months, and where?

$0.

I have self-funded this since April 2026; the first version came out of an Anthropic hackathon. Falsify OÜ was incorporated in Estonia on 12 August 2026.

Four applications are currently pending: EA Funds' Transformative AI Fund ($30k), StandICT (€9.5k, standards work only), GitHub Secure Open Source Fund ($10k), and FLOSS/Fund. No external funding has been received to date.

Commercially, two fixed-scope pilot conversations are open in the €3,000–€7,500 range; no paid pilot has closed yet.

Update, 22 Aug: Two things since posting.

I published the measurement that motivates this project: Singapore's Global AI Assurance Sandbox asks participants in writing to publish their pass thresholds. I read all 29 published case studies from its 2025 and 2026 rounds — a reader can recover the decision rule in 3. Only one case states an explicit numeric threshold. Method and per-document ledger: falsify.dev/notes/threshold-disclosure

Correction, 29 Aug. The sentence above originally read "the one numeric threshold in the corpus exists because a NYC statute required it". That over-attributes it: Local Law 144 compels the audit and the disclosure, not the number — the 80% mark is the EEOC four-fifths benchmark that bias-audit practice applies. It now says only what the ledger supports. I corrected the article as well. On a page about claims fixed in advance, that is worth stating rather than quietly fixing.

And the first warm pilot conversation opened: a European adversarial-audit foundation replied to the question this project exists to answer — how does a third party verify your criteria predated your results?

Update, 29 Aug: Six things since the last update.

Independent convergence. The framing is no longer only mine. NIST's January draft on automated benchmark evaluations compares documenting criteria in advance to scientific preregistration by name. A 48-author frontier-auditing paper recommends preregistering evaluation protocols before results are known. PREP-Eval, a protocol published this year, asks for a time-stamped preregistration. Evaluation Cards goes furthest and mentions hashes of a non-public preregistration. The interesting part is what happens next: Evaluation Cards measured its own preregistration field across 635 benchmarks and found it populated in 0.0% of them. So my claim has narrowed, and I think improved. The idea is being asked for independently. What is missing is the layer that makes it checkable, and adoption of that layer is close to zero.

IANA. I submitted application/vnd.prml+yaml for registration. It is with IANA's designated reviewer and has been posted to the public media-types mailing list. That is not approval.

CycloneDX. The performanceMetric schema in an ML-BOM has no dedicated threshold field and no standard way to reference a pre-commitment. On 28 August I opened PR #193 against the CycloneDX property taxonomy to register a prml namespace so an ML-BOM can reference a receipt. It is an open pull request.

Measurements. The published one is still Singapore: 29 case studies, a reader can recover the decision rule in 3. I have started running the same measurement against an assurance certification scheme, a medical-device regime and a financial-stability document set. I will publish the counts and the per-document ledgers as each is finished; until then I am not using the unpublished numbers as evidence here.

The best objection I have had. One assurance firm told me their managed assessment plan already fixes the criteria before testing. That is the strongest objection so far: the technical gap may be real while the need for an additional verification layer is narrower than I think. The February test is designed to find out.

What I would fund if I were you. The $5,000 independent security review. It is the smallest piece that stands on its own, and it tests the part everything else rests on: the integrity of the commitment path.