Project summary

CrewScore is a deliberately narrow open-source tool for a common failure in AI-agent projects: important safeguards are often discussed, but never written into the instructions that actually configure the agent.

The tool checks whether 23 published controls are explicitly present in a system prompt or instruction file. Those controls cover things such as human approval, prompt-injection handling, cost limits, stop conditions and audit requirements. CrewScore runs offline and is available as a Python CLI, a GitHub Action and a browser checker.

The boundary matters. CrewScore does not prove that a model will obey a safeguard. It is not runtime red-teaming, a security audit or a certification. It is a cheap structural check that can be run before more expensive evaluation.

The project is already public and working. The repository includes the rules, tests, validation harness and a reproducible analysis of 356 publicly collected prompts. Of those, 83 were production-labeled. Their median written-control coverage was 10 out of 100. That result is evidence about what was written in those prompts, not evidence that the associated systems were unsafe.

I am asking for $20,000, with a $10,000 minimum, to spend 12 weeks testing whether CrewScore's individual controls are clear, reproducible and useful enough to keep. The work will be public and MIT-licensed. If a control does not hold up, I will publish that result and narrow or remove it.

What are this project's goals? How will you achieve them?

The goal is to move CrewScore from a maintainer-built checklist to a better-validated public artifact for developers and researchers.

First, I will expand and document the corpus. New material will come from public sources with clear provenance. The release will state how items were collected, what was included or excluded, and why. Coding-agent configuration will not be silently mixed with system prompts; that distinction already caused a misleading analysis once, and the repository now documents the correction.

Second, people other than me will review a substantial labeled sample. I will measure agreement, false positives, false negatives and ambiguous wording at the control level. The current Cost, Compliance and Audit dimensions are explicitly documented as thin. They will receive extra scrutiny rather than being defended because they already exist.

Third, I will publish a control-by-control crosswalk against current OWASP agent-security resources, including the Agent Control Standard and the OWASP Top 10 for Agentic Applications. This will not claim equivalence. It will show which CrewScore controls correspond to established concerns, which are only partial textual proxies, and which need to be rewritten or removed.

Fourth, I will publish evidence for each control rather than leaning on one aggregate score. Each record will include the definition, examples, known failure modes, reviewer disagreement and any rule change. CrewScore already warns people not to rank prompts or vendors by the total score; this work will make the individual-control evidence easier to inspect.

The minimum-funding plan covers the validation protocol, corpus work, independent review, control-level evaluation and a public report. At the full goal I will also add bounded extraction adapters for common agent frameworks such as LangGraph, CrewAI and AutoGen. Those adapters will locate instructions without sending prompt content to a third party. They will not be described as runtime enforcement.

The 12-week plan is:

Weeks 1–2: freeze the protocol and success criteria; publish the draft crosswalk; publish corpus provenance and labeling rules.

Weeks 3–6: expand the corpus; run independent annotation; record disagreement instead of forcing consensus; identify controls with high error or weak construct validity.

Weeks 7–9: publish per-control findings where the data permits measurement; revise, narrow or remove weak controls; rerun the committed evaluation harness; publish every scoring delta.

Weeks 10–12: release the validated ruleset, reproducible corpus artifacts and final report. If the full goal is reached, ship the first framework extraction adapters.

How will this funding be used?

At the $10,000 minimum:

- $6,000 for maintainer research, implementation and reproducibility work.

- $2,500 for independent reviewers and annotation.

- $1,000 for corpus collection, provenance and research support.

- $500 for public reporting, release work and CI reproducibility.

At the $20,000 goal:

- $10,000 for maintainer research, implementation and reproducibility work.

- $4,000 for independent reviewers and annotation.

- $2,000 for corpus collection, provenance and research support.

- $1,500 for public reporting, release work and CI reproducibility.

- $2,500 for framework extraction adapters.

All funded outputs described here will remain public. Funding does not buy a favorable result. Any new or overlapping support received before this proposal closes will be disclosed, and the budget will be adjusted so the same work is not funded twice.

Who is on your team? What's your track record on similar projects?

I am Sarosh Hussain, the creator and current maintainer of CrewScore. I built and maintain the public repository, Python package, CLI, GitHub Action, browser checker, ruleset, validation harness and documentation.

CrewScore is not a grant-dependent concept. It currently ships 23 public controls across eight dimensions, runs locally without an API key, and can be used either as a report-only check or as a CI gate for named controls. The repository includes a security policy, scoring documentation, a reproducible corpus analysis and explicit limitations.

The most relevant track record is not a claim that the project is already validated. It is the willingness to expose mistakes. An earlier scoring scale was broken: a prompt that clearly stated one control in every dimension could still score 28 out of 100. I changed the scoring model and published the measured effect. A later analysis applied governance rules to AGENTS.md-style coding-agent configuration and produced a category error. CrewScore now separates configuration smells from governance coverage instead of hiding that failed approach.

Reviewers paid through this grant will be named in the public project record along with their role and any relevant conflicts.

What are the most likely causes and outcomes if this project fails?

Some of the 23 controls may not be reliable enough for automated detection. Reviewers may disagree, regex rules may create too many false positives or false negatives, or a control may be too context-dependent to score responsibly. In that case the control will be narrowed, marked experimental or removed.

Written safeguards may also have a weak relationship with runtime behavior. CrewScore already assumes that limitation: text presence is not runtime compliance. This project will not turn a structural linter into proof of safety. A useful outcome may be a smaller, better-defined preflight checklist and clearer guidance about when teams need runtime evaluation or red-teaming.

Adoption is another real risk. CrewScore is early and has limited independent GitHub adoption. I am not using funding to support a traction claim. The purpose is public validation and reproducibility. If adoption remains low, the corpus, crosswalk and control-level evidence will still be reusable by others.

Finally, framework-specific extraction can become brittle. That work is a stretch goal and will remain separate from the core validation results.

Negative results will be published rather than reframed as success.

How much money have you raised in the last 12 months, and from where?

CrewScore has not received dedicated external grant funding in the last 12 months. I have funded the work myself. Any support received before this proposal closes will be disclosed, and overlapping scope will be removed from this budget.

Public proof

Repository: https://github.com/shmindmaster/crewscore

Live browser checker: https://crewscore.ai

Python package: https://pypi.org/project/crewscore/

GitHub Action: https://github.com/marketplace/actions/crewscore

CrewScore checks whether safeguards are written down. It does not prove that an agent follows them, replace adversarial testing, or certify a system as safe. This proposal is for making that narrow public artifact more evidence-backed.