You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
$10,000 left to reach the funding goal.
We found that safety training fails to generalise across different phrasing of the same harmful prompt. We tested 18 frontier LLMs on 879 Bengali prompts in our BanglaSafe paper (EMNLP 2026) and found 53.6% of responses were unsafe. The most impactful was seeing a 17 percentage point increase in success rate for the same request in formal newswire phrasing over a normal message.
If we see this framing effect as a general jailbreaking method for various languages, then a single phrasing benchmark is not an effective benchmark. This project identifies and solves this problem.
Test framing effect on several languages with formal and informal phrasing.
Understand why this occurs by analysing refusal rates and responses to prompts.
Train and release open safety judges which generalise across phrasing and languages. English centric judges are missing many harms.
Everything will be made open source including prompts, model outputs, scores and judges.
Answer the question of whether framing shift is a general way to bypass safety with a clear and quantified answer. We will quantify the effect size in each language and model. We will release a benchmark, open judges which outperform English centric judges and a paper.
No salary. All computing and API costs.
$7000 for API calls to frontier models. Many languages x 2 phrasings x ~1000 prompts x ~15 models. Plus API calls to judges.
$2000 for GPU hours to train and test open judges. ~440 H100 GPU hours x $4.50.
$1000 buffer for price fluctuations.
Minimum $3000 so less models and languages will be tested in the core API study. No trained judges.
Shubhashis Roy Dipta, Final year PhD student - Computer Science, UMBC.
Co-author of BanglaSafe ("Register Shifts Break LLM Safety") (EMNLP 2026 Findings). Found this effect.
Worked on safety and robustness benchmarks including: Omni-Modal Dissonance Benchmark (under review at CVPR 2027), TRIAGE (NeurIPS 2026, E&D Track, co-author) and ChannelGuard (preprint, co-author).
First and equal authorship papers at NeurIPS 2026, ACL 2026 and EMNLP 2026. Internships at Amazon (Alexa AI) and Scale AI working on RL post-training/RLHF.
Many open source models and data released on Hugging Face.
The framing effect may only occur in Bengali. This is still a useful finding as it shows where safety teams should look to find issues. The open source multi-language benchmark will still be released.
The frontier models may change during the course of the study. We will track the model versions and make all outputs available.
$20,000 - Google Cloud Gemini Academic Program Award (2026) for agentic LLM research (different project).
EA Funds Transformative AI Fund ($10,000) - Applied
Anthropic External Researcher Access ($1000 API credits).
If these are received I will update this page.
There are no bids on this project.