Project summary

LLM, code and logs (gen10): https://github.com/Aditya-0167/Jarvis-LLM .

Details: https://github.com/Aditya-0167/Jarvis-LLM/blob/main/README.md

Detailed Research paper: https://github.com/Aditya-0167/Jarvis-LLM/blob/main/README-JARVIS_Research_Paper_full_repot.pdf

So I am 16 and I built an research platform for studying whether a small, from scratch self evolving Mixture-of-Experts (MoE) LLM system can identify capability failures, choose an appropriate intervention, learn from targeted information, evaluate whether the change actually helped, and preserve the result as an auditable experimental state and self evolves itself form scratch!

Named it JARVIS and It self evolved itself form 6 million to 156-million-parameter model, trained with no pretrained weights. Its an Auditable Research Loop for Error-Driven Continual Learning in Small From-Scratch Language Models I designed it to save every generation collects public web data retrieves evidence and tests architecture changes. I took it to free Colabs T4 GPU (15GB vram) built it till 10 generations and 3,520 training steps.

Then unfortunately my problem (why i am here) struck me it ran out of GPU vram and without it I cannot proceed!

• My question is simple-
  
  Can a relatively small, from-scratch language-model system identify capability failures, acquire relevant permitted public information, learn from targeted material, evaluate improvement on unseen tasks, and retain beneficial changes over repeated iterations?
  
  • H1 — Error-driven learning: at approximately fixed model size and matched compute, targeted learning from diagnosed capability gaps will yield larger and more reliable held-out gains than generic continuation.
  • H2 — Information access: answer-time retrieval can improve factual performance without changing model parameters; such gains must be credited to information access unless parameter-only transfer is demonstrated.
  • H3 — Retention: an intervention is successful only when its target capability improves while previously validated capabilities remain within a predefined regression tolerance.
  • H4 — Bounded evolution: architecture mutations should be accepted only when resource-feasible and beneficial on held-out evaluation beyond matched continuation.
  • H5 — Evaluation: versioned, unseen benchmarks and regression suites provide a more trustworthy longitudinal signal than raw training loss when the data distribution changes.
  
  These hypotheses are intentionally separable.
  A result can support H2 while failing H1; it can support H1 on one capability while exposing regression elsewhere; or it can show that architecture evolution adds little value after the learning controller is improved. This structure prevents one favorable metric from being treated as evidence for the whole system
  
  Can repeated research actions produce auditable, transferable gains in the underlying system rather than merely better-looking outputs?
  When a small model looks like it improved, did it learn or did it just retrieval a tool or extra training time do the work?
  
  I have the system and a full experiment plan. but all i need is the compute power to run it.

I did all this after leaving a 8 hour of school day and a 3 hour commute for being homeschooled in May 2026. In the months since, I finished Harvard’s CS50x course, built my LLM JARVIS, discovered and reported security bugs to Google and Apple by myself , and wrote two research papers.

Basically, Jarvis is an experimental from scratch model system for studying error-driven continual learning, retrieval, evaluation, checkpoint lineage, and resource-aware model evolution.

What the V6 evidence supports

JARVIS demonstrates an executable, persistent research platform around a small from-scratch language model, including checkpoint lineage, public-web acquisition, answer-time retrieval, evaluation infrastructure, bounded architecture experimentation, and reproducibility controls. The archived trajectory also demonstrates important failure modes: optimization loss can diverge from user-visible capability, retrieval can surface irrelevant evidence, conversational training data can contaminate retrieval when the pipelines are not separated, and resource constraints can block candidate architectures. These observations motivate the controlled V7 program that tests whether error-driven intervention can produce measurable, retained gains under matched conditions.

Neural core:
The archived Generation-10 configuration uses a vocabulary of 256 symbols, dmodel = 384, eight attention heads, 12 layers, seven experts with topk = 4 routing, dropout 0.0, and a block size of 1,280. The tracked parameter count is 156,144,084. The transformer backbone follows the attention formulation of Vaswani et al. (2017), while sparse expert routing follows the mixture-of-experts line represented by Shazeer et al. (2017). This is small relative to contemporary foundation models and is intentionally constrained by single-GPU experimentation.

Information acquisition and retrieval:
JARVIS distinguishes batch acquisition from answer-time retrieval. Batch acquisition changes the future training corpus; answer-time retrieval supplies evidence to the generator without changing parameters. Acquisition records source URL, query, timestamp, source domain, content hash, and filtering decision. Retrieval should separately record query formulation, retrieved documents, source diversity, duplicate filtering, and relevance judgments. This provenance is essential because search is not learning: an answer supported only by a retrieved document should be credited to information access unless a later parameter-only evaluation demonstrates transfer.

Self-refinement and autonomous research:
Self-Refine demonstrates iterative improvement through feedback at inference time (Madaan et al., 2023). The AI Scientist demonstrates a broader loop of idea generation, coding, experimentation, analysis, and paper writing (Lu et al., 2024). These systems motivate recursive loops but also make attribution difficult when multiple mechanisms change together. JARVIS focuses on the complementary methodological problem: how to determine whether a repeated loop improves the underlying model, the routing system, or only the information available at inference time.

System Architecture :
JARVIS is organized as a set of explicit interfaces: neural generation, persistent state, information acquisition, retrieval, deterministic tools, benchmarking, failure diagnosis, and architecture evolution. The controller sits above these mechanisms and selects interventions using the current empirical history. The architecture is modular because modularity is required for scientific attribution.

JARVIS began as an architecture-evolution exercise, but the archived trajectory suggests a more interesting scientific problem. Parameter count is easy to observe, whereas the ability to turn information into durable, transferable capability is harder to measure. The project therefore shifts from “how large can the model become?” to “how efficiently can the system determine what it should change next?”

This shift also changes the role of the model. The neural core becomes one component in a learning system. Retrieval, tools, memory, evaluation, and architecture are not accessories; they are experimental variables. The resulting framework is closer to a small laboratory platform than to a conventional one-shot language-model training pipeline.

For an independent researcher, this framing is practical. Compute constraints cannot be removed, but they can be measured. Failed candidates can become evidence about the feasible design space. Repeated small experiments can yield a richer causal record than a single large training run. The result is a research strategy in which every compute cycle is expected to answer a specific question.

What are this project's goals? How will you achieve them?

• My Goal is to find out if a small model improves more when its training is aimed at the mistakes it actually makes, compared with just training it longer.
  

  1. Generation-10 Baseline:
     Generation 10 is the parent state for the proposed V7 study. The archived record provides a concrete baseline for capacity, optimization history, information accumulation, and lineage. These quantities should be treated as descriptors of one experimental trajectory, not as normalized measures that are directly comparable with unrelated models.
     
     Problem Formulation:
     Let x be a task, M the parameterized model, R the retrieval subsystem, T the deterministic-tool subsystem, and C the controller. The observable response is
     y = F(x; M, R, T, C).
     A change in y does not identify which component caused the change. To measure the components separately, we propose binary indicators U, R, T ∈ {0, 1} for whether a parameter update, retrieval, and tool access are enabled. For a fixed candidate checkpoint and task set, the factorial model is
     y = β0 + βU U + βRR + βT T + βURUR + βUTUT + βRT RT + βURTURT + ϵ.
     The interaction terms are important because retrieval may change which tool is selected, and a parameter update may alter retrieval or tool-routing behavior. We do not assume the coefficients are causal under arbitrary observational data; the design becomes interpretable only when the cells are evaluated under matched tasks, budgets, and randomization or controlled order. The central learning question is likewise conditional. Let B target denote an unseen target set and G a regression set covering previously validated capabilities. A candidate M′ is eligible for retention only when Accept(M′ ) = ⊮ ∆target ≥ δ, ∆reg ≥ −τ, feasible(M′ ), prov(M′ ) .
     
     Here δ and τ are predeclared before the intervention block. Rather than selecting these values after observing a favorable run, we propose calibrating them from pilot variance and a practical minimum effect size, then freezing them for the candidate comparison. The protocol records both the calibration evidence and the final values in the experiment manifest.

• My plan is ->
  
  TO run the Controlled V7 Experimental Design:
  
  The core experimental comparison is a 2 × 2 × 2 factorial design over parameter updates, retrieval, and tools. The design corrects a limitation of a simpler additive attribution story by estimating interaction effects directly
  
  Error-Driven Continual-Learning Algorithm:
  The V7 intervention begins from a frozen parent checkpoint M. The model is evaluated on the target benchmark B_target and the regression suite G. Failures are classified, an intervention is selected, and a candidate M′ is produced under a fixed resource budget. The candidate is then tested on the same target and regression sets.
  
  Accept(M′) = 1 iff Δ_target ≥ δ and Δ_regression ≥ −τ and feasible(M′) = 1 and provenance(M′) = 1.
  
  Here Δ_target is the change in target performance, δ is the minimum practically meaningful gain, Δ_regression is the change in previously validated performance, τ is the maximum tolerated decline, feasible denotes that the candidate satisfies the resource envelope, and provenance denotes that the state and data required to reconstruct the intervention are complete.
  The acceptance rule deliberately separates improvement from feasibility. An intervention that improves the target capability but requires an unavailable amount of GPU memory is not a successful candidate. Likewise, an intervention that improves a narrow target while causing large regressions should not become the new parent state.
  
  This matrix answers several distinct questions. The retrieval-only cell measures information access benefit without parameter change. The parameter-only cell measures whether a learning intervention changes model behavior without external evidence. The integrated cell measures system level capability, while the interaction coefficients estimate whether combinations provide gains beyond their isolated effects. Tool-only and tool enabled cells prevent deterministic computation from being mistaken for neural learning.
  
  The proposed V7 program turns these observations into a controlled research agenda. A frozen parent is evaluated, failures are diagnosed, interventions are budgeted, candidates are tested under factorial attribution and regression controls, and only candidates meeting predeclared improvement, retention, feasibility, and provenance criteria are retained. The result is not a claim that recursive self-improvement has already been achieved. It is a framework intended to make that claim, if it becomes supportable, falsifiable and reproducible

• Test it on a fixed benchmark: arithmetic, science, reading, code, general knowledge, current facts and tool choice. Keep development, validation, regression and final-test sets separate.

• For each failure, name the likely cause: missing knowledge, bad retrieval, reasoning, tool choice, unstable training, forgetting, or hardware limits.

• Run targeted fixes under the same compute budget as the baseline.

• Compare conditions in a 2×2×2 design: parameter update × retrieval × tools. This shows each part’s effect and how they interact. I plan at least 5 random seeds per condition.

• Keep a change only if the target skill improves, older skills do not get worse, it fits the hardware, and the data trail is complete.

• When I audited my first version (V6), I found two problems. Candidate models trained a few extra steps before being compared with the parent, which is unfair. And my data loader re-reads the whole corpus on every step. V7 fixes both before any experiment runs.
  
  

Research paper short: https://github.com/Aditya-0167/Jarvis-LLM/blob/main/RESEARCH%20PAPER%20JARVIS.pdf

Why the controller needs a budget:
Recursive systems have a built-in multiple-testing problem: a controller can repeatedly attempt interventions until a validation score looks favorable. To limit this, each intervention block declares training steps, retrieval calls, candidate count, random seeds where feasible, and a wall-clock or memory budget before evaluation. The final-test set remains untouched until the block is complete. This is especially important when the controller itself is the research subject.

Resource-Aware Architecture Search:
The V6 trajectory demonstrates why resource constraints belong inside the research design. Larger candidate configurations encountered CUDA out-of-memory failures on the available T4-class accelerator.

Such events define the feasible frontier of the search space and can be used to construct a resource model. A candidate should therefore be represented not only by parameter count but also by memory footprint, step time, context length, and checkpoint size. One possible selection objective is improvement per unit compute:

Utility(M′) = Δ_capability / (compute_cost + λ·memory_cost).

The scalar utility is illustrative rather than mandatory. The key requirement is that resource use be recorded and that architecture changes be judged against a fixed computational budget. A candidate that is slightly better but twice as expensive may be scientifically less useful in a constrained setting than a candidate with a smaller gain and dramatically lower cost.

How will you know it worked?

The experiment runs end to end with frozen benchmarks and fixed budgets, and I publish a clear answer. Yes, no and mixed are all valid answers.

And biggest proof it works is V6 loop!

JARVIS demonstrates an executable, persistent research platform around a small from-scratch language model, including checkpoint lineage, public-web acquisition, answer-time retrieval, evaluation infrastructure, bounded architecture experimentation, and reproducibility controls. The archived trajectory also demonstrates important failure modes: optimization loss can diverge from user-visible capability, retrieval can surface irrelevant evidence, conversational training data can contaminate retrieval when the pipelines are not separated, and resource constraints can block candidate architectures. These observations motivate the controlled V7 program that tests whether error-driven intervention can produce measurable, retained gains under matched conditions.

A positive result means targeted learning beats plain continued training on unseen items with retrieval and tools switched off, and older skills stay within a tolerance I set before the run.

I report paired confidence intervals across several random seeds, and I check my automatic scorer by hand on a sample. In an earlier study I found a false positive in my own scorer and fixed it.

Everything is published: benchmark, code, checkpoints, raw outputs and a written report.
Visit: https://github.com/Aditya-0167/Jarvis-LLM .

How will this funding be used?

My experiment needs a machine I control to break past my current 15GB VRAM ceiling. Each tier below is a complete, working setup utilizing massive hardware headroom.

$6,000+ (minimum): MacBook Pro M5 Pro, 64GB unified memory

MacBook Pro M5 Pro, 64GB: about $5,792.80

Taxes: $1,000.

On Apple Silicon, system RAM and GPU VRAM share a single pool through a Unified Memory Architecture. Getting a 64GB pool gives my model over 4x the memory headroom of my current 15GB Colab T4 setup. This gives me the exact headspace I need to scale my context window past 1,280 bytes. I can hold dense activation graphs for multiple random seeds locally without hitting a single CUDA Out-of-Memory failure. I will port my training code straight to Apple’s Metal backend. A laptop runs slower per step than a desktop GPU, but that will not stop me. Putting more time into it is infinitely better than being completely locked out on the free tier.

$9,000+: MacBook Pro M5 Max, 128GB unified memory

MacBook Pro M5 Max, 128GB: about $9,000.

A 128GB unified memory allocation lets the integrated GPU use a massive continuous workspace. This footprint easily beats even a dedicated desktop RTX 5090’s 32GB VRAM limit. This massive memory frontier completely changes my architecture search options. It lets me scale JARVIS candidate mutations all the way up to 1 Billion parameters. I can finally test structural weight internalization at a scale my free T4 cannot run at all.

$13,000+: GPU workstation

If funding reaches $13,000, I will buy the RTX 5090 32GB ($11,374.95) instead of a laptop. It trains much faster and lets me run baseline and candidate experiments at the exact same time.

I need a 128gb (64*2) DDR5 memory kit ($3,686.33), a Samsung 990 pro 4TB NVMe drive ($842.59), and an AMD Ryzen 9 9950X CPU upgrade (about $621.41). I also need a larger power supply (about $400), heavy airflow cooling (about $300), cloud credits ($3,000), and budget for shipping, tax, and import duty (about $2,500). This brings the tier to about $22,725.28.

I already own a Ryzen 5 9600X, an MSI B850 motherboard, 32GB RAM, and a 1TB SSD. I will reuse what is compatible. Any money not spent stays on the same research, and I will publish exactly what I bought and what it produced.

Who is on your team? What's your track record on similar projects?

It is just me, working next to my NIOS studies and family duties. Here is how a 16-year-old got here.

I always wanted to know how machines work. As a kid, I took everything apart—first motors, then an old computer. Back in traditional school, my days were packed. A 7–8 hour school day, extra tuition, and a grueling 3-hour daily commute left zero hours to actually build software. So, in May 2026, I took a massive bet on myself. I pulled out to do homeschooling through NIOS. My parents and relatives completely opposed the idea at first. It took me a full month of showing them stacks of research and technical printouts before they finally agreed to let me try

I saved up my allowance for six straight months just to buy parts and assemble my own PC. My old machine was too weak to even boot Windows 11. Right after that, I knocked out Harvard's CS50x in less than two months, wrapping up with a 95% score on June 1, 2026. That same month, I finished two intensive NASA courses: Open Science 101 and Fundamentals of Remote Sensing.

My schedule is pretty intense. I run around helping at our family shop and managing daily household duties during the daytime. My real study block starts late around 9 PM. I grind until my night's unit is totally finished, usually crashing around 1:00 or 1:30 AM to make sure I get enough sleep to stay healthy.

What I built after that in just four months:

JARVIS. I built it across ten generations: model, checkpoints, lineage tracking and web data collection. I used AI assistants for code and editing, and I take responsibility for all of it. I wrote the V7 experiment plan and a manuscript that separates what I measured from what I only propose. A version is under review at the ARRSI 2026 workshop.

Prompt-injection research. I wrote and ran a 164-trial study on Gemini and found a false positive in my own scorer. Then I built a larger study with 360 cases, six open models and 29,280 outputs.

Security research. I trace deep code paths, find weaknesses, build PoCs, and test them in real environments. For Apple, I learned EXT4, built crafted disk images, and created a macOS crash harness to validate a filesystem vulnerability. Apple reproduced another vulnerability I reported and is fixing there systems and has a planned fix on target for a Fall 2026 release, and has prioritized all my reports for review. I also reported 51 security issues to Google, including multiple P1/P2 findings.

Other work. I launched an homeschooled helper app on Play Store, securing 37% conversion, +224% impressions, & +100% activation growth, and completed the Tata IAM virtual simulation.

Every one of these started with a problem I did not know how to solve, and I learned it by building it.

What are the most likely causes and outcomes if this project fails?

I don’t plan to fail, and I don’t stop. I have kept going every time something broke. JARVIS started on a free Colab GPU, crashed many times, ran out of memory, and still reached ten generations because I rebuilt it and kept going. This project will get the same effort.

The most likely thing that could slow it down is compute, because my free GPU is too small for the full experiment. That is exactly what this funding fixes. If a run still doesn’t work, I will not give up. I will find the cause, fix one thing at a time, and run it again until I have a clear answer.

Whatever the result, the work stays useful. I will publish the benchmark, the code, every checkpoint and the full results, so other people can build on them. I believe in this project, and I will keep working until it is done.

How much money have you raised in the last 12 months, and from where?

None & I have not applied anywhere else yet!