Thanks for the great analysis @RyanKidd!

For the reservations what our perspective on this is:

1. I agree that large AI agent swarms are probably going to be effectively applied in large AI companies first, however the riskiest usages of them seem likely to come from the broader community. With systems like Gastown and Openclaw , with the related Moltbook putting thousands of agents into contact. One of Gastown's big projects right now is trying to connect all the agent swarms they've created into a whole civilization with Wasteland. So, from my perspective, I consider there to be two classes of risk:
- frontier AI lab having a badly run agent swarm causing problems to be associated with superintelligence risk. Because for them to get to the point where that swarm can cause problems it would need to be sufficiently intelligent to outsmart some incredibly paranoid researchers.
- groups of people just trying things out, vibe coding projects and libraries that get used in more complex projects and even companies. Here the risk comes from some important infrastructure relying on memetically compromised code or a few bad agents in the Wasteland figuring out how to misalign the output of thousands.

2. This is much more straightforwardly a risk we are taking. The novelty being that this is a form of alignment that is "passive" in the sense that I can enact a will onto my codebase that is stable even when I am not in control of future agents interacting with it.

3. Also a big worry we've had, and part of why this work might be best suited to be done outside of frontier AI research labs. This work also has the potential to provide methods of insidious "watermarking" type effects to sabatoge competing companies and enhance lock-in (eg. Gemini written code causes Opus to underperform).

4. Our current plan is to do a public benefit corporation and we have been vetting VC's we are talking to closely. We want to encourage more people to research in this space so we will be releasing results publicly, with the actually nasty infohazards themselves being shared with researchers we trust not to misuse them.

Thank you once again for the advice and guidance on how to make the world a better place.