Project summary

LLM forecasters usually state a probability directly. This project tests a different design. A closed vocabulary of 16 cross-domain structures (threshold, constraint, polarity, growth…) is anchored on quoted facts, and a deterministic engine turns the readings into probabilities. Every forecast is sealed (SHA-256) before resolution and scored live against two baselines: the same model alone, and a base rate computed by code. The question is whether a fixed, shared structural vocabulary adds signal beyond the model, and when it should abstain.

What are this project's goals? How will you achieve them?

The goal is to test, live and under seal, whether a fixed structural vocabulary improves LLM forecasts, or whether it only renames what the model already sees. A closed set of 16 cross-domain structures (threshold, constraint, polarity, growth…) is anchored on quoted facts. A deterministic engine then turns each reading into a probability (arm C), which is compared with the same model alone (arm A) and with a base rate computed by code (arm B). Every forecast is sealed with SHA-256 before resolution. The pipeline already runs on Metaculus, and on a text-free bench built on public blockchain data. Funding keeps it running long enough to reach a conclusive sample, adds a reader-reliability study, and pays for publishing the results.

How will you know it worked?

he success criterion was fixed in advance: C beats both A and B on the log score, using anytime-valid confidence sequences, over 200 resolved questions per question type. Secondary measures, also pre-declared: calibration (including the tails), reader invariance across sources, and whether the engine's "surprise" score predicts errors. The project also counts as a success if it shows clearly that C does not beat A. That would be a useful negative result, given that recent papers disagree on whether explicit evidence aggregation helps: Murphy (2026) finds it loses, LEAP (2026) finds it wins.

How will this funding be used?

LLM API costs for 12 months of live forecasting and benches: about $1,600.

A dual-reading study measuring the reader's error rates (sensitivity and false positives): about $1,000.

Part-time development and a written report with all data and code: the rest.

Minimum: $2,000 (API costs plus the reliability study). Goal: $8,000.

Who is on your team? What's your track record on similar projects?

I work alone: Othman Ben Brahim, independent developer in Nîmes, France. I build and publish open-source tools: Python applications, Firefox add-ons published on addons.mozilla.org to https://addons.mozilla.org/fr/firefox/user/19646616 PHP/MySQL web platforms, and a set of Claude skills for structured analysis and forecasting. This project has three public MIT repositories with tests and a versioned changelog:

github.com/Othman-Benbrahim/alphabet-situationnel (engine, 175 self-tests)

github.com/Othman-Benbrahim/metac-bot-template (Metaculus bot, 130 tests)

github.com/Othman-Benbrahim/alphabet-chaine (text-free bench)

I also contribute to EnVrai?, a team project fact-checking political claims ahead of the 2027 French presidential election.

What are the most likely causes and outcomes if this project fails?

The most likely failure is that C does not beat A: the structures only re-express what the model already knows. The registry is designed to show this, and I would publish it. A second risk is reader unreliability. If the model anchors different structures depending on the source, C inherits that noise, and the invariance test measures it. A third risk is a sample too small to conclude anything, because of API limits; that is what the funding addresses. In every case, the code, the sealed registry and the evaluation tooling remain public and reusable.

How much money have you raised in the last 12 months, and from where?

None. The project has been self-funded.