Project summary

There are problems in frontier AI Safety benchmarks for VLMs as they can often exploit vision shortcuts to win on static benchmarks but failing to remain robust to small changes. Exhaustive fixed-length benchmarks can also cost large amounts by wasting API Calls.

I already developed and open-sourced a project called VLM - TRACE. In the coming 6 months, I am planning to extend this framework to include extra features, like combining it with anytime valid sequential hypothesis engine, mentioned by recently peer reviewed paper for IEEE ICICAIEECE 2027. Planning to install as a python installable library and also as a plug-in for softwares like inspect_ai. By adding these technologies together, it is possible to know safety degradation in VLMs and shortcut exploitation.

What are this project's goals? How will you achieve them?

My main goal is making an open-source VLM auditing library and inspect_ai plug-in that any evaluator can run without spending huge amounts on fixed-length API sweeps.

In January 2027, I will take the Innovation E-Detector math from my IEEE paper and build the core e-audit python package, testing it on synthetic drift streams to make sure false alarms stay under 0.05. In February and March, I will extend VLM - TRACE on RunPod A100 and L40S GPUs across four open-weight VLM families (Qwen2.5-VL, LLaVA-OneVision, InternVL, and PaliGemma). I will run white-box PGD + EOT perturbations, Flux and SDXL image edits, background shuffles, and Indic language shifts comparing English with Malayalam, Tamil, and Hindi.

By late March, I am planning to wrap both parts into an inspect_evals solver and scorer plug-in. From April to June, I will run black-box visual prompt injection tests on OpenAI, Anthropic, and Google models, get external red-team spot checks to find where the sequential detector breaks, and upload the Docker setup, .eval logs, and paper draft to arXiv and Zenodo.

How will you know it worked?

I will check four clear things by June 30, 2027:

• The e-audit stopping engine cuts the number of needed evaluation samples by at least 40% compared to fixed-length runs, while keeping false alarms under 0.05.

• Both e-audit and VLM - TRACE are public under an MIT license with passing tests, and a pull request is submitted to UK AISI's inspect_evals.

• Benchmark tables and raw .eval logs for four open-weight VLM families and three frontier API providers are uploaded on Hugging Face Hub.

• A full paper showing both working results and failure cases (similar to my earlier Zenodo and VLM - TRACE reports) is posted on arXiv and submitted to NeurIPS 2027.

How will this funding be used?

I am asking for $20,000 USD for 6 months (January to June 2027), split between my full-time researcher stipend and invoiced compute and validation costs:

• $10,200 for Full-Time Researcher Stipend ($1,700 per month for 6 months working 35 to 40 hours a week from Trivandrum, India, which also covers retroactive work done on the VLM - TRACE release).

• $3,300 for Frontier VLM and LLM API Credits and Evaluation Tooling ($550 per month for 6 months across OpenAI, Anthropic, and Google APIs for visual prompt injection sweeps, multilingual testing, and coding harness automation, all invoiced).

• $2,000 for Contingent IIT Bombay Mentorship or Institutional Fee (kept strictly for IIT Bombay project onboarding fees if confirmed; I will only withdraw this with an official invoice, and if the slot closes before January 31, 2027, the $2,000 will not be drawn or will go to extra API sweeps).

• $1,900 for External Red-Team Audit, Hosting and Dissemination (spot-checking edge cases, Hugging Face storage, Weights & Biases logging, and workshop travel/registration).

• $1,800 for Cloud GPU Compute on RunPod ($300 per month for 6 months giving around 2,700+ hours on A100 80GB at $1.19/hr, L40S 48GB at $0.79/hr, and RTX 4090 at $0.34/hr plus storage, invoiced).

• $800 for Wire/FX Fees and Compute Buffer (4% reserve for international transfer fees and compute overruns).

Planning to draw the grant in two tranches in January 2027 and April 2027, and any unused compute or mentorship amount will stay in Manifund or be returned.

Who is on your team? What's your track record on similar projects?

I am doing this project alone as the principal investigator. I am a final-year B.Tech Computer Science and Engineering (AI) student at Mar Baselios College of Engineering and Technology in Trivandrum, India.

For VLM robustness, I built and open-sourced VLM - TRACE ([insert GitHub URL]) with automated test suites and benchmark plots, and I also merged a pull request (PR #3 fixing BF16 precision) into UBC TEA Lab's vlm-probe repo for counterfactual VLM shortcut testing. For sequential statistics, I am the first author of "Drift-Aware Anytime-Valid Monitoring of Training Integrity: The Innovation E-Detector and Calibrated Detection from the Training Loss", accepted at IEEE ICICAIEECE-2027, and I uploaded a multi-seed federated learning stress test on Zenodo reporting null results openly. I also serve as Chairperson of the IET MBCET Student Chapter and spoke on a panel at the IET International Festival of Volunteering in London in September 2026.

I have contacted faculty at IIT Bombay for external academic mentorship, which needs institutional confirmation and a fee. Since they have not said okke yet and the slot could get missed, I am also applying to SPAR Spring 2027 and AI Safety India's advising program so my research feedback does not depend on one place.

What are the most likely causes and outcomes if this project fails?

The main technical risk is heavy-tailed VLM noise slowing down the martingale evidence accumulation, making sample savings smaller than 20% compared to fixed-length benchmarks. If that happens in the first two months, I will publish the exact negative results so other researchers know where e-values struggle on noisy multimodal data, and use Months 4 to 6 compute to expand the counterfactual shortcut and visual injection dataset.

Another risk is missing the IIT Bombay mentorship slot before funds arrive. Since I designed the entire library and benchmark pipeline to run independently, missing that slot will not delay any release, and the $2,000 fee will simply remain unused. Also, evaluation tools often get ignored when they need custom wrappers, so building directly for UK AISI's inspect_ai lets teams run everything through standard CLI commands.

How much money have you raised in the last 12 months, and from where?

$0 USD.