Project summary

Continuous AI Evaluation (CAIE), an open-source framework that tracks changes in AI systems in real time and determines whether the evaluation evidence supporting them remains current, needs review or is stale. I developed the project and published its technical specification and implementation on IEEE SA Open Source. Project: https://caie.ieee-saopen.org/

What are this project's goals? How will you achieve them?

I want this project to become a shared safety infrastructure for evaluating advanced AI systems and tool-using agent. By rigorously validating CAIE, publishing reusable open-source tools and benchmarks, and building integrations that enable AI safety researchers to adopt it and extend it across agent ecossystem.

How will you know it worked?

Success will be measures by testing CAIE against controlled changes to AI agents, models, tools and dependencies, then comparing its assessments with independent repeat evaluations. I will measure missed material changes, unnecessary reassessments, detection reliability and reproducibility. the ultimate evidence of success will be an independently reviewed, reproducible framework that other AI safety researchers can use.

How will this funding be used?

Funding will support my full-time work developing and validating CAIE, computing and API costs for experiments, controlled testing infrastructure, independent technical reviews and pilot testing. it will also support publishing the open-source benchmark, research findings and implementation documentation.

Who is on your team? What's your track record on similar projects?

I am currently the sole creator and developer of CAIE. I developed its concept, architecture, technical specification, documentation and python reference implementation, which is published through IEEE SA Open Source. I have not yet established a formal team. I intend to engage more AI safety researchers and evaluation specialists to strengthen and test the work.

What are the most likely causes and outcomes if this project fails?

The main risks are limited visibility into externally hosted systems, inaccurate impact assessments, and difficultly demonstrating that CAIE's results generalize across agent architectures. Adoption may also be slower that expected.
I will address these through explicit visibility limitations, independent comparisons, controlled experiments and narrow, evidence led validation. even if the broader approach proves impractical, the experiments will produce useful evidence about the limits of CAIE and publicly document what works and what does not.

How much money have you raised in the last 12 months, and from where?

Actually this is the first platform I have am seeking for funding.