Measuring AI-era legal fabrication that passes every citation check — in binding Indonesian court records.
In a civil case in Indonesia, the defendants’ counsel cited three real Supreme Court decisions — every one of them genuine and verifiable — and attributed to them holdings those decisions do not contain. Two were inheritance-land disputes and one a bankruptcy matter. Each was re-characterized to fit the party’s position and stacked together under a claim that the three were “one hundred percent identical”. The citations exist. What they are said to hold is fabricated.
This is the fabrication a citation check cannot catch. Tools built to detect AI hallucination verify that a cited case exists — and this passes, because the cases are real. The fabrication lives in the holding, which no existence check reads. It can be caught only by opening each cited decision and reading whether it says what is claimed.
We have one confirmed specimen, found under a preregistered sweep (osf.io/5gwej). This project measures how common the pattern is in binding Indonesian court records: real citations carrying fabricated holdings — in short, numerus verus, ratio ficta: a real number, a fabricated rule.
Goal. To measure, with auditable evidence, how often real legal authorities are cited in binding Indonesian court records while being made to say what they do not — and to show that this contamination, invisible to existence-based detection, is measurable, and therefore governable.
Method, and why it is a discovery rather than an accusation:
(1) The measurement protocol is preregistered on the Open Science Framework before the data is examined, so the finding is itself reproducible and fixed in advance, not selected after the fact.
(2) A published, open conformance standard — the Standard of Examinability, a yardstick anyone can apply [DOI 10.5281/zenodo.23662790] — is used as the measuring instrument against a defined corpus of Indonesian civil (perdata) court decisions and the authorities cited within them.
(3) Every flagged citation is closed to its primary source — the cited decision is opened and read — before it is counted, and each specimen is reported against its source, never against the judge.
Attribution stays within the evidence. Where a fabrication carries the internal signatures of machine generation — real case numbers, holdings re-characterized to fit, false precision such as the claim that three decisions are “one hundred percent identical”, authorities stacked across unrelated fields — it is labelled “suspected machine-generated, by internal signature,” never “proven AI.” We do not claim to know which tool produced it.
The outputs are a public dataset and a report, in Indonesian and English.
Why this matters for structural AI risk. The binding records that institutions rely on are becoming a soft target for undetected corruption of exactly this kind — plausible, confident, attached to real authorities, and invisible to the detectors built to catch AI error. And the risk compounds rather than stays still: as AI systems increasingly draw on and cite binding legal records, a fabrication that stands undetected does not merely persist, it can propagate into the material future systems learn from. This is an epistemic-integrity question for the institutions — courts — that societies rely on to remain trustworthy as AI scales.
It is also, as yet, ungoverned. A published study measuring eleven AI-governance instruments — national and cross-border, across the profession and the judiciary — finds that none imposes, in enforceable form, a duty to verify the existence and accuracy of the authorities cited in a legal filing [DOI 10.5281/zenodo.21818675]. The duty is not being evaded; it has not been written. Indonesia is the testbed; the pattern is universal. Making it measurable is the first step to governing it.
The protocol is preregistered before the data is read, so success is defined in advance, not after. The project has worked when the full corpus of 500 civil decisions has been examined and every flagged citation closed to its primary source; when the prevalence of the pattern — real citations carrying fabricated holdings — is reported with its denominator, whatever the number turns out to be; and when the examination is reproducible against the public OSF registration. The deliverables are a public dataset and a bilingual report. A low rate is still a result: the method and the open standard stand as reusable infrastructure either way.
This contamination is invisible to the tools built to catch AI error: it passes every existence check, and it does not surface through keyword or metadata search. It can be found only by a trained reader opening each cited decision in full. This grant funds precisely that labor — the human examination that is the only method capable of detecting the pattern.
A bounded first measurement, sized for a fast grant:
• Corpus assembly and access to court decisions, obtainable in full: USD 2,000
• Structured examination of 500 civil decisions against the Standard of Examinability — opening and reading each cited authority: USD 9,000
• Source-verification and holding-comparison for every flagged citation: USD 2,500
• Preregistration, public dataset hosting, and the report in two languages: USD 1,500
Total requested: USD 15,000, for a registered corpus of N = 500 civil decisions. A larger sweep (N = 1,000) with a second examiner for inter-rater reliability would run to about USD 25,000.
Tandry Laksana is a practicing advocate at Justa Causa Law Firm in Jakarta and the founder of the field AI-Based Jurimetrics. His advantage for this project is specific: a lawyer who reads court decisions in daily practice, and who built the examination method being applied to them.
Published, with registered and preregistered discipline:
• The Standard of Examinability — the open conformance standard this project uses as its instrument [DOI 10.5281/zenodo.23662790].
• AI-Based Jurimetrics: The Declaration and Foundation of a New Field in Legal Analysis [DOI 10.5281/zenodo.20726213].
• A registered provenance study of sixteen fields, with this field as the tested protagonist, its protocol fixed before the data was read [DOI 10.5281/zenodo.24373146; preregistered at osf.io/fzewt].
• A preregistered corpus examination of fabricated citations in peer-reviewed legal scholarship — the same method, registered before execution [preregistered at osf.io/khgpx].
• The Sidoarjo specimen (PN Sidoarjo No. 31/Pdt.G/2026/PN Sda): the first confirmed instance found in his ongoing preregistered sweep of fictitious and misplaced jurisprudence citations, documented in a published, preregistered essay [DOI 10.5281/zenodo.23654864; osf.io/5gwej].
He works in Indonesian; every English output is accompanied by a full Indonesian version he ratifies line by line. His collaboration with AI is disclosed openly — the point being: examine the work, not its origin.
• Prevalence could turn out low. The pattern is rare and labor-intensive to find — our own preregistered sweep confirms it surfaces only through full reading, not automated search. Even so, a single real citation made to carry a fabricated holding, standing in a binding record, is significant, and the method and standard remain as reusable infrastructure for any jurisdiction.
• Corpus access could be narrower than hoped. We scope to decisions obtainable in full and closeable to source.
• Mis-measurement is the real risk. We mitigate it by preregistering the protocol before seeing the data, and by closing every flagged citation to its primary source before it is counted.
• Naming decisions carries reputational and legal sensitivity. We mitigate it by reporting systemically, never naming individuals, presenting each specimen closed to its source, and labelling attribution only within the evidence. This is discovery, not accusation.
One application is under evaluation: the Transformative AI Fund (EA Funds), for the framework-and-standard side of this work. This Manifund project is the evidence side — a distinct, concrete measurement of the pattern — not a duplicate of that application; if both proceed, they reinforce each other. No other funding has been raised.
There are no bids on this project.
Comments
tandrylaksana
19 minutes agoA note on the banner above: yes — this proposal was written in openly disclosed human–AI collaboration, and an AI-text classifier flags it as such. That is not a defect to hide; it is the subject. This field asks institutions to examine the work rather than detect its origin — because origin-detection is unreliable, and, as this project argues, the wrong remedy. Every claim here is closed to a primary source, every figure to a published DOI; I set the direction and bear the conclusions. Examine the work, not its origin.