Project summary

AI safety benchmarks decide whether models are judged safe to deploy, but almost nobody checks whether the benchmarks themselves measure what they claim. AISafetyBenchExplorer is a database-backed platform cataloguing 200+ safety benchmarks with their metric definitions, provenance, citation history, and known limitations, so that researchers can see what a safety claim rests on. It is currently at pre-launch stage; a public API and the field's first longitudinal meta-analysis of benchmark validity are the next deliverables. This grant covers the compute and hosting to get it live and to finish the field's first longitudinal meta-analysis of benchmark validity.

A paper draft of my initial insight from the project can be downloaded here: https://tinyurl.com/AiSafetyBenchExplorer-Report

A sample worksheet of benchmark catalogue which is transitioning into a fully-fletched web-based (not yet live) app is here: https://tinyurl.com/AISafetyBenchExplorer-Workbook

Github code repository access can be provided if requested.

What are this project's goals? How will you achieve them?

The goals of this project are as follows:

1. Traceability: connect benchmark claims to paper, metric definitions, evidence locations, repositories, dates, and versioned metadata rather than treating a benchmark name or score as self-explanatory.

2. Pluralism in validation: create room to evaluate not only numerical performance, but also methodological rigour, construct validity, representational scope, limitations, maintenance, and human disagreement. The idea is to ensure that these dimensions remain distinct rather than being collapsed into one score.

3. Independent scrutiny: support issue reporting, improvement suggestions, expert review, audit trails, and controlled ontology annotations.

4. Reproducible comparison: create a future evaluation layer that preserves the exact model, benchmark version, prompt, decoding settings, tools, evaluators, metric implementations, and run environment needed to understand a result.

Three deliverables are envisioned over six months:

1. public launch of the platform with API and research exports

2. an ontology pilot where independent reviewers annotate 40-50 benchmarks, with agreement data published

3. a meta-analysis of the catalogue with details such as; which safety constructs get tested, which stay under-tested, and whether evaluation methods are improving or converging on convenient proxies.

The platform already exists (PostgreSQL/FastAPI/Next.js, extraction pipelines, review workflows). It only needs further module extension, curation, validation and launch, not building from the scratch.

How will this funding be used?

The funding is intended to be used majorly for compute, infrastructure, and stipend for few expert reviewers for a pilot annotation pipeline validation. The compute fund is for token credits to handle LLM-assisted extraction runs across benchmark papers. Based on my experience working on this project, the quality of extractions by open-weight models is significantly low as compared to closed models. A single extraction job uses three different model parliament frameworks: two extracts, and one serves as a check for correctness and harmonise accordingly. ~ $330/month for compute. I have been paying for credits personally so far. Within the 6 months, on a monthly basis, every AI safety related benchmark papers/releases will be tracked and catalogued, and a re-extraction workflow for existing benchmark paper versions, So, the same operational workflow continues to apply.

Citation and metadata refresh via Semantic Scholar, Crossref, Github, and HuggingFace scraping API functions as a weekly scheduled cron job. Therefore, production and platform hosting, including Redis/Celery broker, for monitoring and backups needs substantial funds. As this project is envisioned to evolve over the years, the budget for platform hosting needs to be adequately projected. Also because the project stays live beyond the proposed 6 months of funding, I am projecting a budget of about $2,000 for this.

There's an ontological annotation framework that automatically builds from safety construct and its attendants facet scope/context wired into the operational pipelines. It is initially inferred from the LLM extraction pipeline (duly labeled for transparency as "LLM Proposed" ), and then a few iteration of human-expert reviews follow for provenance, confidence assessment and validations. These iterations also include contestation. agreement/disagreement, rejection, and acceptance. All these review steps are published as part of the annotation audit logs, with reviewers identifier as well. 30-40 benchmarks will undergo these reviews as a pilot, not only to validate the quality of extractions, but also to improve on the future versions of the ontology. 4 expert reviewers (10 benchmarks each) will be enrolled for this task and a $500 stipend budget for each of them.

Break down: Compute/Credit ($330 x 6 = $1,980) + Hosting ($2,000) + Reviewers (4 x $500 = $2,000) = $5,980. Anything above this, up to the $8,000 maximum, buffers cost overruns, e.g., extraction re-runs when paper are revised, API price change, and higher-tan-expected token use during launch, and any remainder extends computes beyond the six months.

Who is on your team? What's your track record on similar projects?

Just me, for now. I am a Researcher and Data Scientist at CeADAR (Ireland's Center for Applied AI - University College Dublin), where I build AI systems for different domains, such as Law, Health, Finance, etc. I have a Dual Ph.D (Computer Science, University of Luxembourg; Legal Informatics, University of Bologna), with 250+ citations across different peer-reviewed papers. My research interest is at the intersection of AI and Law, especially in high-stake domains such as criminal justice system. I designed and built everything in this project so far; including the catalogue, the extraction pipeline, the review and ontology workflow, and the paper. Although, I plan to onboard few other Ph.D. level researchers for the benchmarks' human-review pilot part, only if/when funding is available. Google Scholar: https://scholar.google.com/citations?user=xy5v9UAAAAAJ&hl=en

What are the most likely causes and outcomes if this project fails?

The realistic failure mode as envisaged is not technical because the platform works. However, the curation and reviewer coordination may take longer than planned while I work part-time, so the meta-analysis could slip. If that happens, the catalogue and platform will still launch, regardless, but what the project seeks to address only takes longer to deliver. In addition, in case of fail, the reviewer recruitment pilot phase risks a potential fallback. In which case, I will target fewer reviewers with more benchmarks each.

The worst case is that the work continues at my personal expense, albeit, months slower. Hence, the project will not die if this grant fails; it only crawls.

How much money have you raised in the last 12 months, and from where?

None. Everything so far, including credits for LLM extraction, has come out of my own pocket. Although, an initial, cli-based version of this project which ran on Ollama leveraged my organisation's internal servers. That's no more sustainable as the pipeline now requires, at least, three frontier models for its operations.