You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
Sparse Autoencoders (SAEs) are a method for unsupervised discovery of concepts from LLMs that have shown promise for their role in interpretability and control. A standard SAE learns a set of sparse one-dimensional signals, but these techniques struggle with data containing hierarchical or multidimensional data [Chanin et al., 2024; Till, 2024; Engels et al. 2024]. In response, SAEs that extract concept manifolds have been proposed using structured sparsity priors, and their efficacy for extraction of meaningful features [Geiger et al.,2026; Fel et al.,2026 ] and control [Wurgaft et al.,2026] have been demonstrated. However, it seems likely that LLMs encode features satisfying all sorts of structural relationships, including but not limited to trees, hierarchies, or multidimensional codes. In this project, we propose to explore parameterisingthe space of structured sparsity priors.
Learning in such a model would extract not only the sparse concept code, but also the relational structure between concepts from data. We plan to try a few methods, including the approach tried in earlier work on image patches [Gregor et al. 2011]. We aim to test validity of each parameterisation using simple toy models, evaluate their efficacy on LLMs, and finally mathematically analyse the resulting optimisation problems to clarify their inductive biases [William Dorrell, 2026]. In sum, we hope to search for meaningfully structured concepts within LLM representations that can then be used for more effective understanding and control.
The funding will be used to support the intern's stipend and other internship-related expenses to allow them to work full-time on the project.
Lahatra is an excellent student with a strong background in mathematics, software engineering, computer science/AI, and more recently computational neuroscience. He is going to use this project to build his skills in mechanistic intepretability, hoping to eventually transition into the field.
Will is an independent research fellow at Harvard who has published in physics, neuroscience, and machine learning. Most of his work concerns theory of neural coding and computation, mainly motivated by animal brains, but more recently he has applied similar approaches to SAEs.
This project hopes to develop new concept extraction techniques. Will will act as mentor, providing his knowledge of the research process and neural coding theories. Lahatra will use this project as a learning opportunity, and to allow him to submit competive applications to PhD programmes in the area.
We have various ideas on how to code up a learnable concept extraction technique but, put simply, they might not work. This will likely be because we make the model too flexible allowing it to collapse in various ways. Despite this, it will hopefully serve as a useful lesson for Lahatra’s training either way, meaning that even in scientific failure, it will hopefully permit Lahatra to get onto his desired PhD programme.
No money has been raised so far.