What progress have you made since your last update?

We put out a paper (https://arxiv.org/abs/2410.04332), LessWrong post (https://www.lesswrong.com/posts/nLRKKCTtwQgvozLTN/gradient-routing-masking-gradients-to-localize-computation), revised the paper during the ICLR review process, and resubmitted to ICML. Manifund's funding was critical for obtaining the results in Table 1 of the paper. The funding also enabled additional experiments that improved our understanding of the challenges and opportunities for applying gradient routing to larger language models. This understanding has shaped our subsequent work for the better.

What are your next steps?

Two of the project members (Alex and Jacob) are writing a research agenda that builds on the original work, as well as advising two projects based on this agenda. We intend to publish the agenda soon.

Is there anything others could help you with?

We have been donating our time to the research agenda and advising. We would be open to receiving funding as compensation for this work (either retroactively or moving forward).

Thanks so much, Neel! This is a huge for the team. Your feedback helped me hone in on key questions that have already changed my sense of research prioritization.

As an aside: I want to make sure it's common knowledge that Joseph Miller, Evžen Wybitul, Jacob Goldman-Wetzler, and Alex Turner share equal credit here for all the hard work they've been putting in behind the scenes.