Can eight measurements make a model intervention more reliable without sacrificing normal task performance?
I study what happens after we make a small change inside a language model. My coauthor Wan Shen and I released a Bayesian method that makes a context-specific prediction from eight measurements. Across six frozen models from four families, it tracked a fixed logit target better than a zero-measurement prior on both evaluation panels. Against a same-budget patch, the comparison was positive on all six in-distribution panels but only two shifted panels. That is a result about tracking, not proof of safer behavior. Code and artifacts: https://github.com/axobase001/dark-subspace-dynamics
What I will test
On held-out synthetic records, can an intervention help the model omit a fictional protected field while correctly returning the field it is allowed to share? I will include matched benign tasks so that a blanket refusal or omission does not count as a win. The comparison will include no intervention, a plain prompt reminder, the zero-measurement prior, and the matched eight-measurement patch. I will freeze the checkpoints, split, and scoring rules before examining test outcomes.
What counts as success
I will report prediction error, observed disclosure, and ordinary task accuracy separately. A useful result requires fewer synthetic disclosures and no more than the prespecified acceptable drop in ordinary task accuracy. If the proxy improves but behavior does not, I will report that plainly. I plan to release the protocol and reproducible aggregate results, including nulls. This is a bounded test on synthetic data, not a replacement for enforcing access controls outside the model.
Budget
The full US$2,500 plan allows US$1,000 for high-memory GPU rental, US$450 for inference and evaluation, and US$1,050 for research software and tools. With US$1,000 I would run a smaller one-checkpoint pilot and report where it stops, rather than promise the full generalization sweep. Parallel funding requests are pending. If another source covers any of this work, I would disclose it and adjust the request; no cost will be charged twice.
Team and track record
I am the applicant and coauthored the six-model study linked above. The prior work includes code, saved artifacts, and numerical replays. The proposed work tests the behavioral result that those artifacts do not establish.
There are no bids on this project.
Comments
No comments yet. Sign in to create one!