You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
Current LLMs don't think, they retrieve patterns. Their reliance on O(N^2) memory makes reasoning a static lookup process. This makes them inefficient and, crucially, uninterpretable, we cannot see why a model hallucinated.
I am building the Recursive-Latent-Matrix (RLM), a recurrent architecture that replaces static embeddings with Dynamic Synchronization Maps.
Mechanism: Instead of fixed vectors, meaning emerges when groups of neurons fire synchronously over time (temporal binding).
Result: The model forms ad-hoc semantic graphs for each input.
Benefit: This reduces memory complexity to O(1) (using fixed-size matrices) and makes the thought process visually traceable as a trajectory in latent space.
I have conducted initial ablation studies on the AG News dataset comparing RLT (Transformer-based) vs. RLM (Matrix-based) architectures.
The Collapse Finding: We found that adding a rigid gating mechanism (Gated RLT) maximizes classification accuracy (88.17%) but degrades generation quality.
The RLM Solution: The Recurrent Latent Matrix (RLM) demonstrates a superior balance. While maintaining high semantic separation, its continuous dynamics prevent manifold collapse, allowing for richer text generation capabilities compared to the gated baseline.
Visualization: Our PCA projections of latent states confirm that RLM produces curved, evolving thought trajectories, whereas standard models exhibit linear, jumpy convergence.
To validate the RLM architecture beyond simple classification, I conducted tests on the WikiText-103 benchmark to evaluate long-context modeling capabilities.
1. Quantitative Stability:
Validation Loss: The model achieved a stable Validation Loss of 4.35 (comparable to ~77 PPL), significantly outperforming the Gated-RLT baseline which suffered from mode collapse.
Compression Efficiency: In preliminary compression tasks, the synchronization map successfully bottlenecked the input sequence (compressing 1024 words to 16 vectors) while retaining enough context to reconstruct text with high syntactic fidelity.
2. Qualitative Generation (Selected Samples): Unlike standard RNNs that often lose coherence, the RLM demonstrates Abstract Association, the ability to maintain a consistent thought vector over time.
Input: "According to the theory of..." RLM Output: "...the first to be created by an artificial intelligence system." (Demonstrates handling of abstract concepts).
Input: "The weather in London is..." RLM Output: "...the first such storm of the 20th century." (Demonstrates contextual consistency and flow).
My goal is to prove that recurrence + synchronization can outperform attention in reasoning tasks while remaining interpretable.
Generative Proof: Scale from wikitext-103 with small batch to coherent text generation on the TinyStories dataset.
Internal Adversarial Reasoning: Implement an internal Critic mechanism (inspired by GANs) to filter hallucinations before output generation. This solves the echo chamber effect in recursive models.
Methodology: I have already built the core RLM and RLT architectures in PyTorch and demonstrated semantic clustering on initial tests. I will use the funding to scale training runs on H100s, moving from "toy problems" to foundation model benchmarks.
The primary bottleneck is Compute.
$5,000: Cloud GPU credits (Lambda Labs / RunPod) for training on TinyStories and partially on RedPajama (1B tokens).
If minimum funding ($1,000) is reached: Proof-of-Concept training on TinyStories (approx. 2 weeks on 1x H100).
Timur Shakur (Lead Researcher & Developer) I am a 17-year-old AI researcher based in Almaty, Kazakhstan.
Engineering: Full-stack developer; built custom LLM agents as Co-Lead Engineer at Diffuzio and Lead AI Instructor at Devyatka LLC (Gov-partnered).
Research: Springer Author; work recognized by Forbes Kazakhstan and the UK Metascience Unit.
I am working solo but actively engaging with the open-source community to refine the architecture.
Since we have already solved the stability issues common to RNNs (vanishing/exploding gradients) via stabilized gating and normalization, the primary remaining risks are fundamental:
The Compression Bottleneck Hypothesis: Transformers maintain perfect memory via the KV Cache O(N), whereas RLM compresses infinite context into a fixed-size matrix O(1).
Risk: It is possible that for high-fidelity reasoning, lossless retrieval is strictly required, and the RLM's compression might saturate on long contexts, leading to forgetting crucial details that a Transformer would retain.
Scaling Laws: While the architecture performs well on small benchmarks (AG News, WikiText-103), recurrent dynamics might not scale as predictably as Attention when trained on billions of tokens.
Outcome: If RLM fails to match Transformer performance, the project will still yield a critical negative result: defining the exact limits of fixed-state memory for reasoning. This would strongly suggest that future AGI architectures must be Hybrid (Recurrent cores + External Memory), rather than purely recurrent or purely attentional.
$0. This project is currently self-funded.