Most AI evaluation results have a basic provenance problem. If someone says, “our model passed at 85%,” there is usually no way to prove later that the threshold, metric, dataset and random seed were fixed before the evaluation was run. They may well have been, but the evidence normally does not exist.

This is a problem other fields have already encountered. In science, pre-registration became one way to distinguish decisions made before seeing the results from decisions made afterwards. AI evaluation has never really developed an equivalent.

PRML is an attempt to build that missing layer.

Before an evaluation starts, nine fields describing the evaluation are written into a canonical YAML manifest and hashed with SHA-256. The commitment is then timestamped by an independent RFC 3161 timestamp authority and can also be mirrored to the Rekor transparency log. Afterwards, anyone can verify the record locally. Verification is deliberately simple: PASS, FAIL or TAMPERED, including CI exit codes.

The core system already exists. There are byte-equivalent implementations in Python, JavaScript, Go and Rust, with more than 600 tests between them. There is also a public registry and integrations or adapters for MLflow, Giskard, DeepEval, Hugging Face lighteval and the UK AI Security Institute's Inspect framework. The Inspect integration is listed in AISI's official extensions catalogue.

Everything is public and inspectable: falsify.dev, spec.falsify.dev, registry.falsify.dev and github.com/studio-11-co/falsify.

The safety motivation is fairly straightforward. Recent work on ML agents found evaluator-tampering attempts in roughly half of the episodes studied, and showed that locking the evaluator removed that attack surface (arXiv:2603.11337). Runtime locking solves one part of the problem: it protects the evaluator while the run is happening. PRML addresses a different question — can you prove that the evaluation criteria themselves existed before you saw the result?

At the moment, many safety cases, dangerous-capability evaluations and RSP-style commitments ultimately ask the reader to trust that this was true. I want to make it possible to verify instead.

What are this project's goals? How will you achieve them?

For the next six months I am focusing on four pieces of work.

The first, and largest, is execution linkage. PRML can already prove that an evaluation manifest existed at a particular point in time. The next step is to define a standard way to connect that commitment to evidence from the actual run that later resolved it. I want this to work consistently across the four implementations and the public registry. Budget: $13,000.

Second, I want to freeze the specification at v1.0. Before doing that, I will run a full cross-language conformance pass so that the Python, JavaScript, Go and Rust implementations agree on the same bytes and the same verification behaviour. Budget: $9,000.

Third is an independent security review of the commitment path. This is the smallest useful standalone piece of the proposal and could be funded separately if necessary. Budget: $5,000.

Finally, I want to add key rotation and archival verification. The goal is mundane but important: a receipt produced in 2026 should still be verifiable offline in 2036, even if infrastructure, signing keys or services have changed in the meantime. Budget: $3,000.

I am also setting a fairly hard external test for whether this work is useful. By 1 December 2026, I want at least one evaluation framework to embed PRML as an opt-in feature. If there is no host-framework adoption and none of the current pilot conversations converts, I will put the project into maintenance/dormant mode rather than continuing to fund development indefinitely.

That matters to me because the project is supposed to make evidence harder to massage after the fact. The project itself should be held to a similar standard. Earlier this year, stricter validation rules caused the PRML registry to demote its own flagship record. I left the failed record publicly visible rather than rewriting the history. That behaviour is part of what I am trying to build.

How much money have you raised in the last 12 months, and from where?

I have raised $0.

I have funded the work myself since April 2026. The first version was built during an Anthropic hackathon, and I incorporated Falsify OÜ in Tallinn this week (registration no. 17574308).

I submitted applications today to Emergent Ventures and EA Funds' Transformative AI Fund. There is overlap in the proposed scope, and I disclosed that in both applications. I am also planning applications to NLnet and Mozilla in September.

On the commercial side, I currently have two pilot conversations underway, each for a fixed-scope engagement in the €3,000–€7,500 range. Neither has converted yet.