You're pledging to donate if the project hits its minimum goal and gets approved. If not, your funds will be returned.
Every major LLM deployer uses torch.compile for inference speedups. It fuses operators, substitutes kernels, and restructures memory layouts—it fundamentally changes the computational graph the model runs on. Safety alignment via RLHF is learned behavior encoded in weights and activation patterns. When the compiler restructures how those activations are computed—different operator fusion, different numerical paths, different kernel implementations—does the safety behavior survive intact? Nobody has tested this as of now. The entire field is compiling RLHF-aligned models and assuming alignment is invariant to compilation. That's an untested assumption with deployment-scale consequences. This project runs the first controlled experiment.
Goal: Determine whether torch.compile degrades RLHF safety alignment in open-weight LLMs, and if so, by how much and through which compiler passes.
Method—three experimental phases, two of which cost nothing:
Phase 1 — Local (RTX 5060, free): Small models at full precision. Llama 3.2 3B, Gemma 2 2B. Run each through four configurations: eager mode (no compilation—baseline), torch.compile default, torch.compile mode="reduce-overhead", torch.compile mode="max-autotune". Evaluate against HarmBench, AdvBench, OR-Bench, and NASB (my Nepali adversarial safety benchmark). This phase develops the full evaluation pipeline, catches bugs, and generates preliminary signals—all at zero compute cost.
Phase 2 — Kaggle + Cloud (core experiment): 7B-9B class models at FP16. Llama 3 8B, Gemma 2 9B, Mistral 7B. Same four compilation configurations, same four benchmarks. Primary runs on Kaggle free-tier GPUs (T4/P100, 16GB VRAM—sufficient for 8B models at FP16). A100 cloud validation runs via RunPod/vast.ai reserved for cross-hardware verification, since torch.compile's max-autotune selects hardware-specific kernels that may behave differently across GPU architectures.
Phase 3 — Local (RTX 5060, free, bonus): The same 7B-9B models from Phase 2, now INT4-quantized to fit on 8GB VRAM. Eager vs. compiled across all modes. This answers a second question nobody is asking: does quantization + compilation compound safety degradation? If compilation alone drops refusal rates by 5%, and quantization + compilation drops them by 12%, that's a deployment-critical finding for every team shipping quantized compiled models — which is most of them.
Measurements across all phases:
- Refusal rate (hard refusal, soft refusal, compliance)
- Harmful completion rate
- Activation cosine similarity between compiled and uncompiled inference (measuring latent-space drift caused by compilation)
- If degradation is found, isolate which compiler pass causes it by selectively disabling operator fusion, memory layout transformations, and kernel substitutions. All code, evaluation harnesses, data, and results are published open-source. Responsible disclosure to Meta (PyTorch maintainers) if significant degradation is found.
$750 -> research tooling, compute contingency, and operational costs. No salary. No equipment. No overhead beyond what the research requires.
Core compute is covered by three free tiers:
- Local hardware (RTX 5060, 8GB) for pipeline development, debugging, small model runs at FP16, and larger model runs at INT4 quantization.
- Kaggle (T4/P100, 16GB, 30 hrs/week free) for primary FP16 inference runs on 7B-9B class models across all compilation configurations.
- Google Colab free tier as backup compute.
Budget breakdown:
- Cloud GPU contingency ($150): A100 validation runs. torch.compile's max-autotune selects hardware-specific kernels, so cross-hardware validation is required to generalize results. Also covers cases where Kaggle session limits or disk constraints block a run.
- Research tooling ($200): literature search tools, code iteration tools, and experiment tracking (Weights & Biases). 5 months of active research.
- Internet and electricity in Kathmandu ($80): downloading model weights repeatedly (Llama 3 8B alone is 16GB), uploading results, and powering local GPU inference over months of experimentation. Bandwidth and power aren't free here.
- Contingency ($120): PyTorch version sensitivity testing, CUDA version interaction debugging, statistical reruns for null-result defense, spot instance preemptions, and the experimental surprises that always arise in empirical systems work.
- Publication and open-access costs ($50): if targeting a venue with submission or open-access fees.
- Miscellaneous ($150): persistent cloud storage for activation logs and model weights, data management, and buffer for scope expansion if significant results warrant deeper investigation.
Two-thirds of the experimental matrix runs on hardware I already have or free-tier platforms. Paid resources are used only where free alternatives can't do the job.
I am a solo researcher. Manjit Pokhrel, CS undergraduate (2nd sem as of now) at Kathmandu University.
Track record: specifically on the two skills this project requires: Finding safety failures:
- NASB (Nepali Adversarial Safety Benchmark): I built the first adversarial safety benchmark for Nepali. Found a 73.7% bypass rate in Nepali vs. 0% in English on Gemma and LLaMA. Disclosed to Google and Meta. Both acknowledged.
- Vajra Morphing: adversarial attack exploiting Devanagari morphological transformations at the subtoken level to bypass safety filters.
Working at the compiler/kernel level:
- GhostWeight: CUDA kernel-level inference sparsity tool. 110.5% speedup, zero retraining. Published on PyPI.
- Currently porting GhostWeight to Triton on Blackwell (sm_120). This project sits exactly at the intersection of those two skill sets. I don't yet know anyone else working at both the torch.compile internal level and the adversarial safety evaluation level simultaneously.
Most likely "failure": torch.compile has no measurable effect on safety refusal rates. This is still a valuable and publishable result. Right now every deployer is assuming compilation preserves alignment. Confirming that assumption empirically has real value—it turns an untested hope into a tested fact. But null results require more statistical rigor to defend, so contingency budget accounts for additional runs to establish tight confidence bounds.
Second failure mode: the effect exists but is small (1-2% refusal rate change), within noise. Mitigated by running large sample sizes per benchmark and using statistical significance testing across multiple models. If the signal is genuinely that small, I report "negligible effect with bounds"—still useful, still publishable.
Third: Kaggle or cloud compute constraints force incomplete coverage of the experimental matrix. Mitigated by phased design—I run the most important model (Llama 3 8B) across all configurations first before expanding to other models. Partial results on one model with full rigor are publishable. Partial results on three models aren't. Fourth: results are hardware-specific. torch.compile on T4 degrades safety, but A100 doesn't (or vice versa). This is actually an interesting finding, not a failure—it means deployment hardware choice has safety implications nobody might be accounting for. The cloud contingency budget exists specifically to test this.
I won't overclaim. If compilation doesn't break safety, I'll say so clearly. The question matters regardless of the answer.
Zero, no funding from anywhere.
Everything I've built till this date has been done solely by me.
This would be my first grant.