Project summary

We train small transformers from scratch where a measurement of the machine doing the computation is fed back into that same computation, inside the matrix multiply that produced it. Concretely: we read the cycle count and cache misses for each k-block of a matmul with rdpmc, and use them as a multiplicative gain on that matmul's own output.

The task is built so that a model with no access to the signal has a known best loss, ln 4 = 1.3863 nats, because the label's position and prefix are drawn independently of the label. That means we can check our controls instead of trusting them. Feeding the counters in lowers next-token cross-entropy by 1.1227 ± 0.0281 nats over three seeds (p = 2.1e-4), and a second machine gives 1.1136. The deaf control, with the signal zeroed, comes out at 1.3895, just above the ceiling where a blind model has to be. That number is the calibration check for the whole experiment, and it is what caught two earlier versions of the run that had a leak in them.

The control we care about most is the yoked arm, which replays the same signal from an earlier step, so that the distribution and the magnitude and the dimensionality are all correct and the only thing wrong with it is the moment it came from. It sits on the ceiling at all three seeds. So what the model uses is timing, not statistics.

It also survives closing the loop. We built a 5M-parameter organism that reads its own counters, picks an action of 0 to 6 cores that really runs and really heats the package, pays its upkeep in the seconds its own choices control, and dies at a real temperature. Self-report, hidden disturbances and work-priced actions all run at the same time. Four seeds, two machines, same ceiling: live 0.621 ± 0.045, clockonly 1.111 ± 0.112, yoked 1.523 ± 0.042, deaf 1.519 ± 0.031. Every control loses in every seed (paired p = 0.004, 2.5e-5, 1.9e-5), and the two arms that ought to land above the ceiling do. The coupling holds while the system is disturbing its own sensor by acting.

We also have a null we can defend. A report head reads only a 32-dimensional interoceptive bottleneck, detached from the trunk so that no gradient from the report can shape the state it is describing, and it is scored against a hidden disturbance that never enters any channel an external decoder could reach. Over 26 arm-runs and 15,600 scored turns on two machines it sits at 0.4997 ± 0.0186 against a chance level of 0.5, z = -0.06. We built the thing so that faking would be impossible, and then it had nothing to say. We could have left it out. Given how much of this area runs on claims nobody outside the lab can check, a null we cannot fake is worth more to us than another positive.

What we are asking for is the funding to turn this into something an outsider can check: a preprint, and a replication kit someone else can actually run.

What are this project's goals? How will you achieve them?

Goal: make substrate coupling something other people can measure and replicate, with the instruments public, and find out whether it survives off our own bench.

Six-month deliverables:

1. A preprint covering the coupling, the yoked-timing control, the closed loop and the self-report null. The numbers exist. What is left is writing it and getting it torn apart.

2. A replication kit that runs with one command. This matters more to us than the paper does. The result lives on hardware behaviour, so the first fair question anyone will ask is whether it works on someone else's machine. We already have a second-machine replication at 1.1136; the work is making it runnable by a stranger, with pinned versions, fixed seeds, and expected numbers with tolerances.

3. The instrumentation, written up and released. AVX2 and RDNA3 WMMA GEMM kernels with per-k-block counter readout, bit-identical to numpy. A minimal perf_event_open/rdpmc reader that is actually correct, which took us several attempts. A thermal safety harness. And a two-part fix that takes AMD gfx1151 (Strix Halo) from about one usable performance counter to 139, checked by decoding a four-way workload at 1.0000 against a 0.24 shuffled-label floor and a 0.20 no-neighbour control. As far as we can tell nobody has published this, and AMD will close the gap eventually, so it is worth doing now.

4. One result outside the toy setting. Does any of this survive with a small pretrained model on a real task? A partial effect is fine and a clear boundary is fine. Either way the question stops being deferred.

5. Two outside adversarial reviews, asked for and answered in public.

Why we think it is worth doing: in the physical neural network, analog in-memory and physics-aware-training literatures, the substrate signal always comes in through the signal path. The state of the substrate itself is treated either as drift to calibrate away, or as a knob the experimenter sets. Adaptive-compute work uses simulated cost. Work with hardware counters feeds the measurements to a network running somewhere else. We have not found anyone training a model from scratch where its own contention and timing enter its own matmul, with an objective that gives it a reason to use them.

How will this funding be used?

Salary for the six months the package takes, for two people in Sweden. Compute is close to free: the three machines we work on are on loan from HP, and we are separately applying for EuroHPC Benchmark/Development Access.

Roughly 85% salary, 10% publication and outside review, and 5% for a third machine or cloud hours if the generality check needs hardware we do not have. No overheads and no institution taking a cut, since we do not have one.

Who is on your team? What's your track record on similar projects?

Two people in Sweden, no affiliation, no previous grants. Eight months of full-time work on this, 812 commits.

We do not have a publication list. What we can show instead is a record of catching and killing our own results, which for a claim like this one is probably the more useful thing to look at.

- Over one 48-hour stretch we found ten instrument bugs. Seven of them had been producing the same comfortable null. Three of our own conclusions went with them, including a reward-hacking story that turned out to be a units error: the cost meter was dividing by a denominator the agent itself chose.

- Two runs of the central experiment were thrown out by their own validity gate before the one we report, because a scoring bug had put the deaf control 0.024 nats below a ceiling it cannot legitimately beat.

- We pre-register and then actually use it. Before the last arm of one battery ran, we wrote down the sentence we would publish if the primary comparison failed. It failed, so that is the sentence that got written, together with the reason, which is that we had built the primary comparison on the least stable control in the set. A longer run at four seeds later passed the same comparison. Both are in the record, in that order.

- Runs that got contaminated are kept in directories named after whatever went wrong, for instance org1_clockonly_s2_ran_beside_my_own_measurement.json.

- An outside audit of our repository this month turned up two gaps we had missed: a set of headline numbers with no output files behind them, and dry-run outputs that had persisted PASS verdicts. Both are flagged in place now, and neither touches anything quoted above, which all has files behind it.

Whatever else is true, we are not going to be the ones who oversell this.

What are the most likely causes and outcomes if this project fails?

We think the likeliest failure is that people find the result trivial: you put a thermometer inside a network and the network used it, and calling that sensor a body does no scientific work.

The answer we have is that a thermometer's statistics alone would be enough, and they are not, since the yoked arm sits on the ceiling, and that a reading taken from outside would not survive closing the loop, where the sensor gets contaminated by the reader's own actions. Whether that settles it we honestly do not know. It is what outside review is for, and part of why we want money to expose the work instead of extending it.

After that: the effect could be specific to our silicon. We have two machines and one replication. A third would settle it, and the kit is meant so that anyone can check.

Or the generality result comes back empty and this only ever works in synthetic worlds at 5M parameters. Then we publish it as empty.

In all three cases what exists at the end is a controlled and honestly reported piece of work, plus open instrumentation other people can use. That includes the gfx1151 counter unlock, which is useful to anyone doing performance work on that chip whatever they make of our claims. For this amount of money that seems like an acceptable floor.

We have also set ourselves a kill date. If there is no funding, no host lab and nobody engaging with it by a fixed point, we write the whole thing up and stop.

How much money have you raised in the last 12 months, and from where?

Zero. No grants, no revenue, no investment. The only outside support is three machines on loan from HP.