Update, 25 October: SDCP is now being checked against reality, plus two corrections
Since the proposal I have moved part of the work from my own synthetic worlds to tests I do not control.
1. Live forecasting under a pre-registered rule. Since 18 October an AI forecasting bot that follows the SDCP rules (every forecast logged before the outcome, nothing rewritten) has posted 69 forecasts in the Metaculus FutureEval Fall 2026 and MiniBench tournaments, and 13 in Market Pulse. Reality resolves them, not me. On 23 October, before any outcome was visible, I locked a calibration test (M1): measure whether the bot's 80% intervals are too narrow on resolved questions, freeze a correction, and test it only on new questions. The lock includes my own predictions (for example, P = 0.70 that the bot is overconfident). A separate review session read the evaluation code and found seven issues; all were fixed and recorded before any result was opened. First read-out: 2 November. It will be published whatever it shows.
2. Real physics data next. A shadow tournament on NASA ACE / OMNI solar-wind data is being designed: 72-hour forecasts sealed by hash before the data arrives, scored the same way as the FDL Benchlab Solar Wind Prediction 2026 tournament.
Two corrections to the proposal:
- Evaluator independence is weaker than I wrote. The evaluator from a second AI company has not been available since 11 October. Reviews are now done by a separate Claude session, the same model family. I report this as a real weakness, not a detail.
- Funding: Emergent Ventures declined on 12 October. EA Funds is still pending.
The minimum ($2,400) still covers six months of model usage for the generator and the reviewer.