TRACE: An Open Verification and Attribution Layer for AI-Enabled Science

TRACE applies software quality assurance principles to scientific claims: reproducible evidence, independent review, clear attribution, and transparent funding.

Why I started TRACE

I have spent years working in software quality assurance, including testing banking systems, APIs, integrations, and data flows.

In QA, a confident statement is not enough. If someone says that a system works, we still need evidence: requirements, logs, test conditions, versions, reproducible steps, and an independent check.

While experimenting with AI systems, I began to notice a similar problem in science.

AI can now generate hypotheses, summaries, analyses, and biological claims very quickly. But checking whether a claim is supported by the underlying evidence still takes significant human effort.

A scientific conclusion may depend on a particular dataset, cohort definition, statistical assumption, software version, or interpretation choice. Those dependencies are often difficult to see from the final conclusion alone.

TRACE began as a question:

Could some of the discipline used in software QA help make the verification of scientific claims more transparent?

I am not proposing that software testing can replace scientific expertise. It cannot.

TRACE is an attempt to build infrastructure that helps scientists, reviewers, developers, and funders see how a claim was evaluated, what remains uncertain, who contributed, and how the verification work was funded.

This is an early experiment, not a finished platform.

What TRACE does

TRACE turns one bounded scientific claim into a documented verification workflow:

claim → evidence → reproducible analysis → independent review → receipt

The final output is called a Scientific Payment Receipt.

It records:

• the original wording of the claim;

• the data and publications used;

• relevant code and software versions;

• important assumptions;

• supporting and contradictory evidence;

• methodological limitations;

• contributor roles;

• reviewer findings;

• unresolved disagreements;

• allocation of project funding.

The receipt is not a certificate of scientific truth.

It is closer to an audit trail. It shows what was checked, how it was checked, who checked it, and what could not be established.

The pilot

I am seeking $15,000 to complete one 12-week pilot using public, legally reusable, non-sensitive biomedical data.

The pilot will focus on one published biomarker–outcome or genetic-variant association.

The exact case will be selected with a qualified biological advisor. It must be narrow enough to reproduce computationally and must not involve direct clinical decision-making.

The project will examine how the interpretation of the selected claim changes when we test factors such as:

• cohort definitions;

• statistical assumptions;

• data-quality conditions;

• analysis choices;

• correlation-versus-causation boundaries.

The goal is not to build a scientific marketplace in 12 weeks.

The goal is to complete one honest end-to-end test of the TRACE workflow.

What the pilot will produce

The pilot has four core deliverables.

1. One reproducible scientific case

We will select one published claim, freeze its wording before analysis begins, and attempt to reproduce the relevant computational result.

The public case will document:

• the selected claim;

• the source publication;

• the public dataset;

• the analysis steps;

• the software environment;

• major assumptions;

• results that support the claim;

• results that weaken or contradict it;

• remaining uncertainty.

Any changes to the original claim will be recorded rather than silently replacing the initial question.

2. One Evidence Pack

The Evidence Pack will bring the material needed to understand the verification into one place.

It will include the relevant sources, data provenance, analysis code, methodological decisions, limitations, and reviewer comments.

The purpose is not to collect every paper related to the subject. It is to make the reasoning behind one bounded verification process inspectable.

3. Two independent reviews

At least two people who did not perform the original analysis will review the work.

Where possible, one reviewer will focus on the biological interpretation and another on methodology, statistics, or reproducibility.

Reviewers will be paid for completing a structured review. Their payment will not depend on approving the original claim.

If reviewers disagree, the disagreement will remain visible.

4. One Scientific Payment Receipt and minimal prototype

The project will publish a simple machine-readable and human-readable receipt showing:

• what work was performed;

• who performed it;

• which evidence and versions were used;

• which problems were found;

• whether the analysis could be reproduced;

• how the interpretation changed;

• how project funds were allocated.

A minimal open-source prototype will support the basic workflow of registering a claim, attaching evidence, recording contributor work, adding reviews, and generating the receipt.

Stretch goals

If the core pilot is completed within the available time and budget, additional work may include:

• a reusable TRACE specification;

• a small benchmark for future pilots;

• external testing of the receipt schema;

• a second example claim;

• integration with existing open-science tools.

These are stretch goals, not requirements for calling the pilot complete.

Why include payments?

Scientific verification requires real work.

Researchers review literature. Developers reproduce code. Data curators check provenance. Statisticians inspect assumptions. Domain experts evaluate whether a conclusion is biologically reasonable.

This work is valuable even when the original claim turns out to be weak, inconclusive, or wrong.

The economic principle behind TRACE is:

People should be paid for verifiably reducing uncertainty, not for producing the conclusion preferred by a sponsor.

The pilot will test whether scientific contribution and payment allocation can be documented in the same workflow without creating harmful incentives.

I do not assume that the first attribution model will work well. Discovering that it is too subjective or administratively expensive would be a useful result.

How success will be evaluated

The pilot will be considered successful if:

• one claim completes the full workflow;

• an independent reviewer can rerun the main analysis;

• important statements are linked to evidence or marked as uncertain;

• contradictory and negative findings remain visible;

• two independent reviews are completed;

• contributor roles and accepted work are documented;

• the Scientific Payment Receipt is generated;

• the core specification and prototype are released publicly;

• project spending is connected to documented work or deliverables.

A particularly strong result would be interest from an external research, biotech, funding, or AI-for-science organisation in running a second pilot.

However, continuation is not the only acceptable outcome.

The pilot will also be useful if it shows that:

• the workflow creates too much overhead;

• attribution is too subjective;

• reviewers cannot use the structure consistently;

• the receipt is too complex;

• existing tools already solve most of the problem;

• the cost of the process exceeds the value of the uncertainty reduction.

Those findings will be published rather than reframed as success.

Timeline

Weeks 1–2: Select the case

I will recruit a biological advisor and identify candidate published claims based on public datasets.

Before analysis begins, we will define:

• the exact claim;

• the dataset and licence;

• the scope of reproduction;

• reviewer roles;

• success and stop conditions.

If suitable scientific expertise cannot be recruited, the biological analysis will not begin.

Weeks 3–4: Map the evidence

The project will assemble the main publications, datasets, analysis dependencies, assumptions, and known limitations.

The purpose of this phase is to decide what must actually be checked.

Weeks 5–7: Reproduce and test

The selected analysis will be reproduced as closely as practical.

We will inspect data provenance, code, statistical assumptions, sensitivity to analysis choices, and the difference between association and causal interpretation.

Weeks 8–9: Independent review

Two reviewers will examine the analysis and Evidence Pack.

Errors will be corrected, but unresolved scientific disagreements will remain in the final record.

Weeks 10–11: Generate the receipt

The project will produce the Scientific Payment Receipt, release the minimal prototype, document funding allocation, and prepare the public case.

Week 12: Publish the retrospective

The final report will compare the outcome with the original success criteria.

It will include the time and cost of each stage, the main failures, and a recommendation to continue, redesign, or stop.

Budget

The maximum funding goal is $15,000.

• Scientific contributors and data curation — $3,000

• Independent scientific and methodological reviewers — $2,500

• Prototype and receipt-schema development — $3,500

• Computational analysis and infrastructure — $1,500

• Open documentation and publication — $1,500

• Project coordination — $1,500

• Replacement reviewers and contingency — $1,500

Total — $15,000

At least $5,500 will go directly to scientific contributors and independent reviewers.

If the project receives only the minimum funding amount of $5,000, the scope will be reduced to:

• one frozen claim;

• one reproducible computational analysis;

• one independent review;

• a minimal receipt;

• a public retrospective.

A reduced pilot will not be presented as a completed full validation.

About me and the team

TRACE is being initiated by Alex Safonov, a software QA engineer and product-oriented technical builder.

My background includes:

• testing banking and financial integrations;

• REST API and SQL validation;

• regression and integration testing;

• risk-based testing;

• translating ambiguous requirements into testable conditions;

• defect investigation;

• causal analysis;

• provenance and retrieval workflows;

• traceable AI-agent actions and safety chains.

My role in the pilot will be product design, QA methodology, technical coordination, workflow development, and documentation.

I am not a geneticist, physician, statistician, or biomedical principal investigator.

Because of this, the scientific part of the project will not begin until qualified biological and methodological expertise has been recruited.

Scientific judgments will remain with the relevant experts. TRACE will organise and document the verification process rather than replace them.

Main risks

The first risk is that the project remains too broad.

Evidence mapping, reproduction, review, attribution, and funding could together create an unusable workflow. This is why the pilot is limited to one claim and four core deliverables.

The second risk is that contribution attribution may be subjective.

The project will use predefined tasks, acceptance criteria, fixed minimum payments, and written explanations. Any contribution score will be treated as a practical allocation tool, not an objective measure of scientific worth.

The third risk is administrative overhead.

TRACE may cost more to operate than the value it creates. The pilot will measure the time and cost of each stage and publish those numbers.

The fourth risk is lack of qualified collaborators.

If suitable advisors and reviewers cannot be found, the project will not attempt to produce a biomedical conclusion without them. The scope will be reduced or unused funds will be handled according to the grant agreement.

Potential harm and safeguards

TRACE could cause harm if a formal-looking receipt makes a weak claim appear certified.

For this reason, TRACE will not label a claim as “true,” “approved,” or “scientifically certified.”

The receipt will display uncertainty, limitations, failed checks, and reviewer disagreements prominently.

The pilot will also follow these boundaries:

• only public, non-sensitive data will be used;

• no clinical recommendations will be produced;

• reviewer payment will not depend on approving the claim;

• conflicts of interest will be disclosed;

• sponsors will not control scientific conclusions;

• no automated system will have final scientific authority;

• process quality will be separated from biological truth;

• private genomic or health data will not be collected.

Public benefit

The grant-funded prototype, schema, case study, and documentation will be released openly.

TRACE may eventually support commercial services such as private workspaces, integrations, or managed verification. However, this pilot will not create exclusive ownership over public scientific facts or grant-funded open standards.

Donors will not be promised financial returns.

The purpose of this grant is to test public scientific infrastructure, not to finance a closed commercial product.

Why fund this experiment?

I am not asking funders to assume that TRACE will become a universal scientific standard.

I am asking for support to test a much smaller question:

Can one scientific claim be turned into a transparent and reproducible workflow that preserves uncertainty, separates independent review, records contributor work, and makes project spending auditable?

If the answer is no, the project will publish why.

If the answer is yes, TRACE may provide a useful building block for scientific work in an era when claims can be generated faster than they can be carefully checked.

The requested funding supports the smallest end-to-end experiment that can provide a meaningful answer.