Project summary

I propose a small, public reproducibility study of how random preference-label noise affects Direct Preference Optimization (DPO) on an approximately 1.5B–3B open model. This is proposal-stage independent work, not a claim of a novel method or a demonstrated reduction in frontier AI risk.

What are this project's goals? How will you achieve them?

Compare 0%, 10% and 20% random flips of training preference labels while holding a clean evaluation set and training budgets fixed. Start by selecting appropriately licensed public weights and preference data and profiling a tiny baseline. The measured memory and runtime will determine model/data size, sequence length and feasible repetitions. Run up to three seeds if resources permit; publish any scope reductions. Main experimental work is planned over 6–8 weeks after access is available.

How will you know it worked?

Success means releasing an auditable experiment: exact configurations, seeds and noise-injection procedure; held-out results across the noise conditions; measured GPU consumption; and a write-up stating uncertainty and limitations. A null or inconclusive result still counts as a useful output if it is reproducible. I do not promise improved model quality or claim that this small test establishes broad safety conclusions.

How will this funding be used?

Up to $500 for compute only: a provisional $50 profiling stage, $350 for controlled training comparisons, and $100 for fixed held-out evaluation/reproducibility reruns. These are spending ceilings, not measured quotes. The $500 supports only the experiments that measured costs allow. No salary, hardware purchase or travel is requested. I will reduce the scope rather than incur unapproved costs. Other applications are pending; any award will be disclosed and no experiment will be double funded.

Who is on your team? What's your track record on similar projects?

I am Anjan Bhanuprakash, a recent graduate based in India, working independently with no current enrollment, university/lab affiliation or registered company. I have used Qwen and Gemma and received a bronze medal in Kaggriculture on Kaggle. Public profiles: https://github.com/anjan672 ; https://www.linkedin.com/in/anjan-b-35a884295/ ; https://www.kaggle.com/anjan4646/competitions . These are general profiles, not an existing DPO project repo. No DPO demo, completed results or publications are claimed.

What are the most likely causes and outcomes if this project fails?

The model or training stack may not fit the available GPU, runtime may exceed the provisional budget, or the small study may be too noisy to show a meaningful difference. Profiling comes first so a smaller model, data subset or fewer seeds can be used. I will publish feasibility limits and negative findings rather than overstate an effect. Synthetic random flips are only a narrow proxy for preference-data errors and do not represent all real annotation noise.

How much money have you raised in the last 12 months, and from where?

I have received no grant or sponsorship funding in the past 12 months ($0; no funding sources). Other funding applications are pending, with no awards confirmed. I will disclose any award and avoid double funding the same experiment.

Preparation note: this proposal was drafted with AI assistance from applicant-provided facts and the approved project scope. No experimental results have been generated or claimed.