Existing implementations of impact certificates (like hypercerts, social impact bonds or this ACX mini-grant) struggle with verification and valuation of outcomes. Awarding funding based on fraudulent or outlandish claims can jeopardise the integrity of this financing mechanism.
With this project, I plan to use the evaluation techniques of outcome harvesting and relentless monetisation to:
Identify beneficial outcomes that a nonprofit has helped produce
Calculate the cost incurred to create the outcome
Quantify the benefit to society from the outcome (using formulas developed by Robin Hood foundation: https://www.robinhood.org/what-we-do/metrics/)
By August-end, I will publish our findings on the benefit cost ratio of 3 nonprofits across the domain of environment, nutrition and education in Auroville (India). This project is a small step towards cross-comparability of social programs in different sectors, a necessary prerequisite for a social stock market or an impact token.
Our team consists of qualified researchers with a track record of media articles and papers in top peer reviewed conferences. The project lead Devansh Mehta is an investigative journalist with the ability to go deep into the rabbit hole and emerge with new knowledge.
Impact evaluation of the nonprofit media outlet CGNet Swara and their work during the COVID-19 lockdowns in India
Paper:https://dl.acm.org/doi/10.1145/3530190.3534827
Article: https://idronline.org/article/programme/crisis-relief-where-we-failed-and-what-we-learnt/
Collection and evaluation of data collected in Gondi (a low resource tribal language) to build a machine translation system
Paper: https://arxiv.org/abs/2211.16172
Investigation into an airline with ties to Azerbaijan's dictator receiving Pentagon contracts and loan guarantees from U.S. Export-Import Bank.
Article: https://www.occrp.org/en/investigations/8493-airline-once-linked-to-azerbaijan-s-ruling-family-got-us-loan-guarantees-military-contracts-planes
All peer reviewed publications can be accessed here: https://scholar.google.com/citationsuser=OaX2N8AAAAAJ&hl=en
To make this a success, we are requesting at least $4250 in funding to be allocated in the following manner:
$1250 as a prize for the nonprofit having the highest benefit cost ratio. Adding an element of competition will ensure there is full cooperation (and some gamification) of our evaluation exercise
$3000 in salaries for evaluators from April to August ($600 x 5 months). Verifying and quantifying outcomes is a non-trivial task looking at proof of outcomes created through interviews/records, calculating cost per outcome using past budgets, employing data science techniques to estimate an outcomes benefit to society and considering the counterfactual case (if the nonprofit didn't exist, how much impact would still have happened?).
Devansh Mehta
9 months ago
I did want to write an update on our project, in the hope that our investor Carl is able to turn a profit off the impact certificates he holds.
We have completed a prototype for testing on ethereal testnet : app.voicedeck.org
OP Mainnet launch is sometime this month.
Our open source github repo : https://github.com/VoiceDeck
and a video talking about our completed demo : https://www.youtube.com/watch?v=Mrjz02xVF3s&t=2s
Devansh Mehta
over 1 year ago
I have spent ~$10,000 in the last 6 months, with travel to conferences, design and some development work being the biggest expenses.
I am getting additional support from the following sources
Gitcoin : My project received $280 in community contributions in the beta round, which unlocked nearly $2000 in matching funds. Gitcoin Grants Round 18 ended earlier this week and my project received $500 in community contributions from 75 people, which I estimate will be another $2-3k in matching funds.
I was selected as a next billion fellow by Ethereum Foundation! Among many forms of assistance, they are helping support my travel later this month for Berlin Blockchain Week and the Global Investigative Journalism conference in Gothenburg (Sweden).
The Plurality Institute gave another $2k earlier this year to support the creation of journalism hypercerts.
We are finally at the stage where our blueprint is ready and now need resources to bring it to the world.
Pretty good actually! 3 citizen journalism media outlets in India (CGNet Swara, Video Volunteers and Gram Vaani) want our help converting their past database of 10,000 impact reports into hypercerts on a public blockchain. These impact reports are stories from their network of rural reporters who narrate longstanding community problems, such as absentee government teachers , unpaid wages to laborers & lack of clean water or electricity. These are shared by the newsroom with senior government officials, mainstream media outlets and urban activists ; if they get resolved, it counts as an impact report.
Only yesterday, we completed code enabling these newsrooms (and any web2 nonprofit really) to convert their past database of impact into hypercerts (a web3 primitive for impact certs) on Goerli testnet
https://github.com/hangleang/hypercert-impact-reports
You can see the completed output here https://docs.google.com/spreadsheets/d/1uTXcOefrEmtOOQwsaqprHJ-WMEi3W6tJP7nE6uPIAQc/edit#gid=544952228
We also had to do some thinking around the customer profile for journalism impact certs, as they do not have an obvious customer base like carbon credits. We decided to target the gifting market, to tap into the growing trend of donation cards commemorating a special occasion (such as Save the Children's Gift Catalogue.)
In our design, hypercerts remain locked when there are 0 contributors. The first contributor gets their congratulatory wishes engraved into the cert, while comments from follow-on contributors are ordered underneath the engraving according to most recent and amount given. After the cost to create that impact is met, the hypercert is again locked
Figma flow : https://www.figma.com/file/NDqXubv1Kzi3v27U9By9fb/VoiceDeck?type=design&node-id=0-1&mode=design
To identify cases where an impact cert is double sold, each hypercert has a unique tokenID on a public blockchain. We also give every customer a ENS sub-domain upon registration, with their contributions converted into units of the hypercert & transferred to them on a public blockchain.
YouTube link with explainer : https://www.youtube.com/watch?v=4upwjTiz_Jc
I would say a 8//10, with 2 points subtracted for not completing a prototype and getting a few sales of journalism impact certificates by now.
At the same time, figuring out how to build something is less challenging than knowing what to build in the first place. I am overall happy as we have our figma flow in place alongside a script to convert private databases of impact into hypercerts on a public blockchain. It's now only the implementation details that need to be looked into
Scott offered to let ACX grantees write on his blog - would love to take him up on that after launch!
Some insight on customer segmentation, we haven't yet tested the waters on product-market fit. Would appreciate suggestions on forums we can write our ideas in to get feedback on our design for an impact certificate marketplace.
Presenting at an EA conference early next year (I was rejected from EAG Boston and EAGx Berlin)
When I now look back at my earlier comments/updates, I am amazed at how many pivots we've made. If this were a regular grant, funders would be aghast at how far what we ended up building is from our initial proposal.
Devansh Mehta
almost 2 years ago
Now that we're past the halfway mark, here are some exciting updates:
Product:
Our partners are CGNet Swara and Video Volunteers, community media outlets with a decades worth of experience in creating impact on the ground
Both organizations have collectively generated 5000 impact reports of on-ground community change (see appendix). We have converted these into impact certificates with help from GPT4 to quantify each one's individual impact
We are selling each impact certificate for between $50-$500. You can see a rough prototype on voicedeck.org . 85% of sales go to the organization generating the impact certificate while 15% goes to VoiceDeck for GPT4 evaluation & marketing of certs
Asks
Buy an impact cert when our product is live! Media is severely underfunded relative to other sectors (only 0.3% of global aid) and impact certificates are the best way to fund journalism that creates tangible outcomes
Help get the word out! Past pilots on impact certificates have failed due to lack of buying pressure. We need help getting on podcasts, offers to write blogposts, etc to drive up sales of impact certificates
We are selling 25% of shares in this project for $1.5k, if you think we have potential then snap them up!
Appendix:
Video Volunteers impact database: https://www.videovolunteers.org/category/videos/impact-indiaunheard/
CGNet impact database: https://web.archive.org/web/20230128202137/http://cgnetswara.org/impact2.php
I have spreadsheets of all impact reports and approval from both organizations to market them to customers.
Devansh Mehta
about 2 years ago
Thanks Austin for validating our problem statement and that retroactive impact evaluation is worth investigating :-)
Auroville is a commune in South India of 3000 people from 50+ nationalities, with tons of nonprofits working in nearby local villages. I've identified Sadhana Forest (reforestation) and Auroville's solar kitchen (nutrition) as 2 local NGOs whose impact we can quantify, although this can only be finalised after I shift there next week (my wife got a research posting here for the rest of 2023).
I plan on doing the evaluation myself. I need to take advantage of Auroville's nonprofit density! I'm also excited to apply evaluation frameworks I've been studying like relentless monetisation & outcome harvesting on actual data from nonprofits.
Austin Chen
about 2 years ago
Hi Devansh, I very much think the problem of retroactive impact evaluation is quite difficult and am excited to see people try and tackle the area! It's nice to see that you've already lined up three nonprofits (from your local area?) to assess.
My questions:
Have you already spoken with these nonprofits about assessing their impact? If so, what have their responses been like?
Have you identified the evaluators who will be doing the work of impact assessment? If so, what are their backgrounds like?