You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
UMNA arose from a fairly simple problem.
When a new component is added to an AI system, how can we know that no existing function has been unintentionally damaged?
Assessments of fine-tuning, adapters, and comparable update methods usually depend on whether the new capability improved. This is necessary, yet it leaves the inquiry's second half unanswered: what happened to the behaviours we never meant to change?
UMNA is my attempt to make that question explicit.
Instead of repeatedly modifying one shared system, UMNA arranges capabilities around a fixed core. New capability modules are trained separately. A calibrated gate can decline when uncertainty appears, and a statistical promotion gate tests an update against protected capabilities before acceptance.
The system also keeps a provenance record, so an experimental result can be traced to the protocol and code behind it.
A working proof of concept and public benchmark results already exist. Some results held steady across several runs. One of our hypotheses did not replicate. That result was also published.
A more valuable next step, I believe, is to test these mechanisms on real models, compare them with standard baselines and established benchmarks, and have key experiments replicated independently.
Public work:
github.com/nabvian/umna-benchmarks
This project's central question is:
Is it possible for an AI system to acquire a new capability while also providing convincing proof that previously protected capabilities have not quietly deteriorated?
I am unwilling to base an answer on the existing prototype by itself.
Over the course of the grant, we would begin by fixing the key hypotheses and experimental protocols prior to conducting the final experiments. This method is already part of our current benchmark work, since it reduces the chance of altering an experiment once an unfavourable result appears.
We would subsequently progress from the present proof of concept to genuine, moderately scaled models and standard evaluations.
The experiments would examine UMNA's principal components both individually and in combination: the frozen-core architecture, capability routing, calibrated admission and abstention, the statistical no-regression promotion gate, and the provenance system.
Findings would be contrasted with simpler alternatives, such as conventional fine-tuning, LoRA/adapters, and modular methods, wherever the comparison is technically valid.
What matters is not merely whether UMNA introduces a capability.
We must determine:
Is the new capability functional?
Were protected capabilities altered?
Is the system able to identify a detrimental change?
Does it needlessly turn down benign updates?
Does the outcome hold across differing random seeds?
Does it hold with larger, more realistic models?
Is another individual able to replicate the result using the frozen protocol?
Where suitable, we would rely on established benchmarks instead of devising one merely to flatter our architecture. Possible evaluation sources include subsets of HELM, MMLU, and BIG-bench, along with specially designed capability-regression tests.
Arjun Gaur would independently rerun the experiments and challenge them.
For the key findings, an uninvolved external researcher or engineer should also receive the protocol and attempt an independent replication of UMNA.
Upon project completion, we would publish the validation materials: benchmark harness, protocols, results, baseline comparisons, failure cases, and reproducibility tooling.
Negative results are permissible.
Should any mechanism fail under more rigorous testing, this must be established prior to others building upon it.
A maximum funding target of $59,850 is hereby established for a twelve-month period.
The mainline variant requires approximately $45,150. A minimum variant may be executed for approximately $37,800.
At the maximum funding level, the funds would be allocated roughly as follows:
Koushik Das — Principal Investigator: approximately $25,000
The work encompasses architecture, research design, implementation, experiment design, statistical validation, analysis, and final write-up.
Arjun Gaur — Research and Validation Engineer: approximately $12,000
Arjun's role involves testing, reproduction, debugging, and independently challenging results from the main development work.
Independent external validation and reproduction: approximately $4,000
This funds an arms-length reproduction of key experiments. This matters because an architecture should not be certified solely by its creators.
Compute: approximately $12,000
Training a frontier model from scratch is not proposed.
Data, storage and research software: approximately $3,000
Contingency: approximately $3,850
At present, the core team comprises two individuals.
Koushik Das serves as the principal investigator.
I am responsible for the design of UMNA and for directing its research and architecture.
UMNA has progressed beyond the status of a mere proposal. An experimental system now exists, and benchmark results have been published, supported by preregistered protocols and provenance.
I cite the failed experiments because they reveal something significant about my intended management of this project.
A planner hypothesis initially appeared promising, yet it failed to replicate consistently across repeated runs. Rather than erase it, we retained the result and documented the hypothesis as failed.
I have likewise finished a technical project that was previously funded.
Emergent Ventures granted me $10,000 for PATHEX, a clinical decision-support system for complete blood count reports. The engineering prototype is complete, and the work has advanced to clinician-guided validation alongside the College of Medicine and JNM Hospital in Kalyani, India.
Although UMNA is a separate project, PATHEX taught me how to carry a system from architecture and engineering to a point of evaluation beyond my own development environment.
Arjun Gaur holds the role of Research and Validation Engineer.
Independent testing, reproduction and debugging form his primary duties.
The clearest failure mode: UMNA succeeds on the proof of concept, yet its benefits vanish on real models.
This is feasible.
A second scenario: the no-regression gate functions, but proves overly conservative. Halting every dubious update would technically avert regression, yet yield no useful architecture.
The inverse failure can occur too: the gate admits updates that subsequently harm a protected capability.
Routing may likewise grow unstable as capabilities multiply.
The frozen-core premise may itself prove excessively restrictive. Certain new capabilities may truly demand representational changes on which an older capability also relies.
An additional, more mundane research risk exists: our experiments may inadvertently favour our own architecture.
Hence the project incorporates frozen protocols, standard benchmarks, matched baselines, repeated seeds and independent reproduction.
Should the central hypothesis fail, the response must not be to rename the project and proclaim success.
We would record which mechanism failed, under which conditions, and what evidence refuted the hypothesis.
The benchmark harness, regression tests, protocols and reproduction tooling ought to remain valuable research outputs.
Evidence that seemingly robust no-regression mechanisms break down past a given scale would itself merit attention.
Emergent Ventures awarded me $10,000 for PATHEX, a distinct clinical decision-support initiative.
Those funds were designated for purposes other than UMNA.
Having fulfilled the engineering goal supported by Emergent Ventures, PATHEX progressed to its clinical validation phase.
To date, UMNA has obtained no grant funding, receiving $0.
Development of the present architecture, implementation, and public benchmark work has relied on our personal time and resources.
I am likewise submitting applications to additional research funders on behalf of UMNA. Any funding obtained or overlapping commitments will be disclosed, ensuring that the same work is not supported twice.