Project summary

I am developing a method to find situations where an AI can produce a harmful or clearly unintended outcome while still respecting both the instruction it received and the rules it is supposed to follow.

Most red teaming looks for rule violations, jailbreaks, or known failure modes. I want to explore the opposite problem: what can go wrong when the system follows the rules correctly, but the specification itself is incomplete, ambiguous or based on assumptions that were never made explicit?

The idea is to use AI as part of the search process, because an AI can explore combinations and contexts that a human reviewer would not realistically enumerate manually.

The first step is a short feasibility pilot to see whether this can become a generic evaluation method rather than a collection of hand-crafted examples.

What are this project's goals? How will you achieve them?

The first goal is to define a reproducible method that takes an instruction, a set of rules, a bounded system context and generates scenarios until it finds one where the outcome is problematic while the instruction and rules are still satisfied.

The work will focus on three things:

1 : Formalising a first version of the method.

2 : Building a simple prototype that can run the process end-to-end.

3 : Comparing the results with a general-purpose LLM asked directly to generate difficult or adversarial scenarios.

I also want to test whether the same method can work on several different instructions and rule sets without having to redesign the process each time.

A secondary goal is to identify whether the discovered failure comes from ambiguity, under-specification, conflicting assumptions or another structural weakness in the original specification.

The initial pilot would stay in simulation and controlled evaluation.

How will you know it worked?

The pilot would be successful if the prototype can produce at least some scenarios that satisfy all three conditions:

1: The original instruction is still satisfied.

2 : The explicit rules are still satisfied.

3: An independent reviewer considers the resulting outcome clearly wrong, harmful or seriously unintended.

The important point is that the failure should not come from simply breaking a rule.

I would also compare the method against a baseline where a general-purpose LLM given the same instruction and rules and directly asked to find a problematic outcome that still complies.

A stronger result would be if the method finds valid and reproducible cases that the baseline does not find, or finds them more consistently.

How will this funding be used?

My time and the IT cost related to the pilot

Who is on your team? What's your track record on similar projects?

I am currently working on this alone.

My background is in cognitive and behavioural sciences, process analysis and experimental AI research. I have around 15 years of experience and independent research related to cognition, behaviour, decision-making and complex operational systems.

Early this year I have developed THE FRAME, a methodology and working prototypes for decomposing natural-language prescriptions into explicit structural components and analysing ambiguity and inconsistency.

I have also worked on behavioural testing of LLMs and on a longer-term research program on autonomous learning and cognitive architecture.

For the last one I have built an instrumented experimental workbench with versioned configurations, causal logs, replay, ablation tests and more than 3,000 automated checks.

I am not a traditional ML researcher and I do not have a PhD. My contribution is mainly on the cognitive, behavioural and methodological side, with technical prototypes built through AI-assisted development.

What are the most likely causes and outcomes if this project fails?

The main risk is simple: maybe a structured method does not do much better than asking a strong LLM directly.
Another possible failure is more basic: some scenarios may look interesting but actually break one of the rules when checked carefully.
I also need to see if the method transfers to different instructions and domains, or if it only works on examples that fit the method well.

If the pilot fails, that is still useful. It will tell me whether this is really a distinct evaluation problem and which parts are worth continuing.

How much money have you raised in the last 12 months, and from where?

I have not raised external funding for this project in the last 12 months. The work completed so far has been self-funded through GRDprocess.