Project summary

We know output-level guardrails (such as Llama Guard, post-generation classifiers, regex wrappers) show up late to the party! They inspect text after the forward pass and autoregressive decoding are already complete. That 'diagnostic lag' gives prompt injections, deceptive alignment, and tool-call hijacks a large window to execute before any external filter flags them.

I built Auditor OS to move monitoring directly into the model's residual stream during inference. Instead of reading generated tokens, it measures the activation kinematics of early, mid and late layers in the network during the inference forward pass.

The architecture is split into two functional steps:

1. Stage I (Prefill Audit): Before allocating the KV cache, the system evaluates prompt-sequence derivatives and detects discordance in layer contractions. It aims to identify direct prompt injections before generation begins and to cluster prompt activations to establish baseline representations for benign reasoning.

2. Stage II (Decode Governor): During autoregressive decoding, the system normalises step velocity by token surprisal (Δτ_t) and decomposes acceleration into tangential semantic pacing (a_parallel) and orthogonal shear (a_perp). If the monitored score Ωt reaches or exceeds 0.19, the governor triggers an early-warning response within one to three generated tokens. The response applies top-K logit penalties or drops network traffic using an eBPF XDP_DROP filter

The mathematical framework is documented on Zenodo (DOI: 10.5281/zenodo.23315812), and the open-source PyTorch reference implementation currently has 14 passing unit tests. To date, however, the early-warning results have only been demonstrated on synthetic activation shocks. This grant will fund the compute and development time needed to test whether the approach generalises to 70B-parameter models under real adversarial attacks.

What are this project's goals? How will you achieve them?

I want to run a focused 90-day sprint to take this from an in-repo research kernel to become a tested, open-source safety tool running on top of vLLM.

Month 1: Real-Model Hooking & Baseline Tuning

• Wire the forward hooks in PyTorch into actual transformer layers (layers 4, 16, and 30) on Llama-3.3-70B using dedicated H100 instances.

• Ensure the surprisal metric [Δτ_t = clamp(I(w_t)/I_0, 0.10, 5.0)] and epistemic context centroids calibrate across standard instruction datasets (OpenOrca, UltraFeedback).

• Measure decode overhead: optimise the fused Top-K logit steering code to ensure it adds less than 2 ms of latency per token step. Also verify that harmless complex reasoning doesn't trigger false alarms (target false-positive rate: < 0.5%).

Month 2: Adversarial Red-Teaming & Lead-Time Stress Testing

1. Run systematic adversarial-prompt evaluations using HarmBench and AdvGLUE.

2. Test whether Stage I prefill auditing detects malicious prompts before the first output token is generated.

3. Evaluate Stage II against mid-generation tool-call hijacking and goal drift. Measure the exact lead time: after an adversarial pivot, does orthogonal shear (a_perp) reliably raise Ωt to at least 0.19 within one to three generated tokens?

4. Test whether the macro-drift bound (D_macro > 3.5) detects gradual, multi-turn jailbreak attempts.

Month 3: Production Middleware & Open Release

• Package the Auditor as a drop-in middleware module for vLLM and TensorRT-LLM serving pipelines.

• Finalise the two-layer containment mechanism: terminate routine streams in userspace for low-latency handling, and use an asynchronous eBPF signal to trigger termination of TCP connections when the host is untrusted.

• Hook up the auto-minting of signed .sent evidence receipts (RFC-001, DOI: 10.5281/zenodo.23314793) on every vetoed trace so downstream systems have an unforgeable record of why the model was stopped. 

• Publish the full benchmark dataset, negative findings, and deployment code openly on GitHub and Zenodo.
  

How will this funding be used?

Total request: $45,000 USD (90-day sprint)

• $18,000 — Full-Time Research Runway: $6,000/month for 3 months. This will let me work on this project for 70+ hours a week without taking on consulting or contract work.

• $18,000 — GPU Compute (H100/A100 Instances): Use of on-demand and reserved GPU instances from Lambda Labs or RunPod. Continuous layer-activation extraction, logit-steering sweeps, and automated HarmBench evaluations across 70B-parameter models can quickly drive up GPU compute costs.

• $6,000 — Linux Kernel / eBPF Contract Support: It allows for a short-term contractor budget for an experienced systems engineer. The contractor will help clean up and stress-test the userspace-to-eBPF ring buffer bridge so the socket dropping doesn't cause kernel panic under high concurrency.

• $3,000 — Testbed Hosting & Open Archival: Six months of edge hosting for the public demo portal ( sentinel.thohatventures.com ), hosting for public-key endpoints, and Zenodo dataset storage fees.

Who is on your team? What's your track record on similar projects?

Track Record:

I’m the founder of Thohat Ventures and the architect of Auditor OS. Over the past several months, I have taken the project from an applied maths concept to a functioning codebase, with 14 passing unit tests on GitHub (Regsiren/Auditor-OS-V50-Sentinel), two citable publications on Zenodo (DOIs: 10.5281/zenodo.23315812 and 10.5281/zenodo.23314793), as well as a live interactive demo at sentinel.thohatventures.com. I build systems end to end, from the underlying differential geometry to low-level serialisation and deployment infrastructure.

What are the most likely causes and outcomes if this project fails?

Failure Modes & Learning Points If It All Fails:

• What if benign creative tasks look like adversarial attacks? Creative writing and strongly argumentative responses at times naturally produce large orthogonal deviations (a_perp). If prompt-conditioned context centroids do not adapt well enough to avoid false alarms, I’ll test monitoring only on the final 25% of network layers, where task-related representations may be more stable.

• What if runtime logit steering introduces too much inference latency? If computing top-K unembedding projections during decoding noticeably slows token generation, I’ll disable active steering and retain a passive diagnostic monitor. The monitor will terminate the stream only when Omega_t >= 0.19 which is the threshold for a hard phase collapse.

• Worst-case scenario outcome: Even if the in-memory steering proves too noisy for general deployment, this project produces an open, quantified dataset which maps how internal activations behave during actual jailbreaks across 70B models, plus an open-source PyTorch profiling tool as well as eBPF containment probe that other alignment researchers can use.

How much money have you raised in the last 12 months, and from where?

$0 USD in external funding. The project has been 100% self-funded through personal savings.

That personal investment carried Auditor OS from initial math concepts to its current operational milestone: two citable Zenodo specifications (DOIs: 10.5281/zenodo.23315812, 10.5281/zenodo.23314793), an open-source PyTorch reference implementation with 14 passing tests as well as the live deployment at sentinel.thohatventures.com.

My self-funding runway concludes at the end of this month. I’m requesting $45,000 to support 90 days of full-time development and dedicated H100 compute. The budget includes $18,000 for personal runway at $6,000 per month and $27,000 for compute. The support would let me complete and publish adversarial benchmarks on 70B-parameter models without pausing development to take on commercial contract work.