Project summary

• This is a follow-up to Belief state geometry of language model personas. Using that funding, we have received promising results about models becoming more Bayesian updaters over personas for small models — a monotonic effect in dense models up to 27b.

• It's important to find out whether these scaling laws scale to larger models before making big claims in our paper!

• We're seeking about $20k to run these scaling experiments

What are this project's goals? How will you achieve them?

• Goal: Scale to larger models and evaluate whether scaling laws hold

How will this funding be used?

• All of the funding will be spent on Runpod clusters, mostly 8xH200.

• We're looking to replicate our existing successful experiments on approximately 7 more large models (100B-800B), with scaling runs costing between $1.5k to $4k

• Anything left over from scaling experiments will go to other experiments on the project, e.g. running more complex versions of our existing successful experiment.

Who is on your team? What's your track record on similar projects?

• Team: Me and Thomas Jiralerspong.

• Track record: We have a small-scale version of this paper which we submitted to the NeurReps workshop at NeurIPS 2026 (currently under review) and are working on a submission to ICLR.

  • Paper will be posted to arXiv soon! Still editing...

• Though I'm a relatively new researcher, Thomas has published a lot of good research in the past

What are the most likely causes and outcomes if this project fails?

• Likely 'failure': scaling laws fail to hold. In this case, the rest of the findings remain significant, and we will have learned a useful negative result.
  

How much money have you raised in the last 12 months, and from where?

• I have raised $20k in the past 12 months, $12k from grantmaking.ai and $8k from AISTOF.