Travel Support for NEURIPS PARIS 2026
AI systems today often split a job across several agents. One agent reads the document, another summarizes it, and another uses a tool. A hidden instruction from an attacker in the document can thus pass from agent to agent until one of them acts on it. We looked at this in our paper 'Separating Injection Presence from Executor Compromise in Residual-Stream Probes of Multi-Agent LLM Pipelines'.
We ran 108 attacks against Qwen3-32B. Only 10 were resisted. 48 resulted in a simulated unauthorized tool call, and the other 50 were simply passed through. Adding an intermediary agent reduced the execution rate from 67% to 22%, though the instructions still made it through. Enabling reasoning mode was even worse (31% to 57%). We also tried to detect attacks by inserting probes into the model. The probes would tell you whether an attack was present, even at the very last agent, which had absolutely no exposure to the bad document. They didn't really help predict whether the agent would act on it, though. That's the bit that would matter if you were looking for a monitor that could tell you when to intervene.
Our paper is accepted for presentation at the New in ML Workshop at NeurIPS 2026 in Paris. In addition to the presentation, I'm also hoping to host a small research dinner at the conference. I'm asking for $5,500 to cover the trip and the dinner.
Links:
Paper: https://openreview.net/pdf?id=pOJZaW2o0r
LessWrong post: https://www.lesswrong.com/posts/gLhnc4eDCGzNhkrX3/what-happens-when-a-collusion-probe-only-finds-a-thin-signal
Code: https://github.com/base-research-lab/spec-gap
At the conference, I'm aiming to attend workshops on agent security and more. Two I'm interested in are Foundations of Language Model Security: Theory, Practice and Fundamental Limits and SLM-Agents: Workshop on Small Language Models for Agentic Systems. I'm interested in seeing how other people go about testing security in tool-using agents. I hope to continue this conversation at a curated, invitation-only dinner with 10-15 researchers and industry professionals.
Feedback collected from our poster presentation on December 11th and an updated paper published on arXiv by November 30th. Within 30 days of the conference, we'll post an update here with the feedback we've received, who we've connected with, and at least one research decision that has arisen from the conference.
I’m requesting $5,500. $4,500 is for the trip, and $1,000 is for the dinner.
Round-trip flight, San Francisco Bay Area to Paris: $1,400
Conference registration: $405
Hotel: $1,600
Meals: $400
Airport transfers and local transport: $200
Poster printing: $60
Buffer for travel costs and exchange rate: $435
Research dinner (venue, food, and drinks for 10-15 people): $1,000
The minimum goal is $3,200.
I'm a biomedical scientist transitioning into AI/ML. PhD from OSU and a postdoc at Stanford. My area of interest is AI safety and biosecurity, and I led this project. This would be our group's first time at NeurIPS. We're hoping to meet some collaborators and mentors there.
SPEC-GAP is joint work with Ifeoluwa Jayeola, Oudoum Ali Houmed, Eyiara Oladipo, Elena Ajayi, Chijioke Ubajaka, Anagha Late, and Lawrence Wagner at BASE (Black in AI Safety and Ethics) Research: https://www.baseresearch.org/
Track record:
We started off with a smaller pilot which we wrote up on LessWrong in July. The probe in that pilot barely beat chance. We showed the results to researchers at FAR.AI, which led to the current paper using a larger model and agents that can actually take actions. For the paper, we built the test system in LangGraph and ran it on H200 GPUs via Modal. Code is on GitHub.
The biggest risk is money. If we can't raise sufficient funds or if it's too late, then we won't be able to attend in person. The dinner could also end up with very low turnout given people's schedules that week. We'll be sending out the invitations well in advance. On the research side, our results may not generalize to other models. If they don't, then we'll of course report that too.
$0. This is self-funded and independent.
There are no bids on this project.
Comments
No comments yet. Sign in to create one!