Riesgos Catastróficos Globales

AI governance Biosecurity Global catastrophic risks

Jorge Andrés Torres Celis

Not fundedGrant

$0raised

Project summary

We want to advance four lines of work related to improving global risk management.

First, we want to promote the report we co-wrote with ALLFED on Food Security during Abrupt Sun Reducing Scenarios (ASRS) among policymakers in Argentina and start the process of legislating an emergency plan in these scenarios.

Second, we want to promote our report on improving the implementation of the EU AI Act in the European sandbox in Spain and connect regulators in the sandbox to auditing experts such as ARC Evals and Apollo.

Third, we want to write a report on biological surveillance in Guatemala, highlighting some cost-effective opportunities for engagement such as news surveillance.

Fourth, we want to write a report on improving risk management in Latin America. This ties with our previous work on risk management in Spain, and would focus on the use of cost-benefit analyses as a tool for risk prioritization and analysis.

Project goals

For each of our projects, the goal is to advance the creation of relevant policy and connect policy-makers to relevant experts. We will know we are succeeding in a given project if our reports are used as support material for drafting policies or if we are credited with connecting policy-makers and relevant experts.

SPECIFIC PROJECT GOALS:

FOOD SECURITY
- Goal: The Argentinian government invests in assessment reports and planning guidance to improve the chances of an adequate response to nuclear winter
- How we know we achieved it: report on response strategies incorporated into governmental agencies, the government includes the recommendations in the national five-year risk management plan.
AI
- Goal: Incorporate critical policy recommendations contained in the attached report such as external auditing and red-teaming into Spanish AI legislation, ideally influencing other countries inside Europe via the AI sandbox or even outside Europe through the Brussels effect.
- How we know we achieved it: government promotes the amendments based on our proposals to the EU AI Act inside the European Parliament.
BIORISK
- Goal: Incorporate policy recommendations in the National Epidemiological Surveillance System (SINAVE) of the Guatemala government.
- How we know we achieved it: The health ministry and other government offices promote the modification of the manuals and procedures of SINAVE.
Improving Risk management
- Goal: Promote the improvement of risk management plans at the Latin American level from the perspective of GCR.
- How we know we achieved it: with the integration in the discussion of GCR terminology and the improvement in ECLAC's risk management indicators.

How will this funding be used?

Concept

Percent

Wages

65%

Conferences and travel

4.2%

Other operational expenses (People Operations, Workshop expenses & Miscellaneous)

16.4%

Margin of operations

8.4%

Fiscal sponsorship fee

What is your (team's) track record on similar projects?

Here are some relevant past outputs of the organization:

An investigation of risk management in Spain that culminated in a collaboration agreement with Madrid’s city hall.
A report on Food Security in Argentina in the event of an Abrupt Sunlight Reduction Scenario (ASRS). We are currently discussing with Argentina policy makers the implementation of an emergency plan to mitigate this risk.
A Report on AI regulation for the EU AI Act Spanish sandbox. We have published two excerpts as standalone articles: 1) Proposals for the AI regulatory sandbox in Spain, 2) Survey of specific risks derived from artificial intelligence.
Some other minor outputs include an article about the new US GCR law, and the co-drafting with the Simon Institute of two responses to the policy briefs from the UN’s Our Common Agenda.
Overall, our website sports the largest collection online of GCR articles in Spanish, with over 46,000 visits from Spanish-speaking countries.
Our team has also built a strong network of advisors and collaborators. Interim director Jaime Sevilla is also director of Epoch, and Director board member Juan García works as a research manager at ALLFED. Between them they have ample experience with management, founding organizations and GCR policy. We also have a close relationship with ALLFED, the Simon Institute and CSER (Centre for the Study of Existential Risk).

How could this project be actively harmful?

We could be crowding out other efforts from more trustworthy sources, though we consider this unlikely and we are committed to collaborating with complementary efforts.
Some of our policy recommendations might be harmful and hard to correct. We are however erring towards recommendations with broad support and backed up by other experts in the GCR community, which should diminish the chances. We also focus more on creating connections and facilitating infrastructure for evidence based decision making than in pursuing specific agendas.

What other funding is this person or project getting?

We are currently operating through a grant from Effective Venture’s donor lottery and other individual donations

Joel Becker

over 2 years ago

Hello Jorge and the rest of the Riesgos Catastróficos Globales team!

Here are some early impressions that I have about your proposal:

I am pretty excited about the senior staff. I have worked a little with Juan, and found him to be smart, clear-thinking, resistant to group-think, and focused. My interactions with Jaime are more personal/less professional, but I am impressed with Epoch.
The projects all seem interesting and reasonable, with fairly clear theories of change. It's not difficult for me to imagine these projects leading to valuable policy changes in the targeted countries (nor is it difficult to imagine these changes spreading elsewhere).
That said, when I ask myself "to what degree does the 95th percentile version of these projects improve catastrophic risk outcomes?", I notice that I don't feel like the projects tackle what I understand to be the most important bottlenecks in e.g. AI and bio.
But perhaps I shouldn't be thinking of the projects as providing the primary path to impact. Two alternative paths to impact might go through mentoring junior staff and Spanish-speaking movement building. I feel early optimism about the first of these -- Jaime has a track-record here (Juan might too, I'm just less familiar). I feel more confused about how to think about the value of these kind of projects for movement building, and how to think about the value of successful movement building.

With the above in mind, some questions for you:

Which paths to impact do you see as most important? What does the 95th percentile version look like?
How do you currently see RCG tying in with mentorship and/or movement-building goals? (Are Jaime and Juan devoting time to mentorship, what will opportunities look like for getting involved, what's the profile of person who engages with your outputs and might be interested in contributing themselves, etc.)
How might RCG look if it were optimizing for mentorship and/or movement-building?
Any other reactions to my impressions, places you think I'm mistaken, etc.?

Jorge Andrés Torres Celis

over 2 years ago

Hello Joel,
Thank you for taking the time to read our proposal, here our answers:

1- The most important paths are:

Continue with the strategic partnership and collaborations. This includes working with different stakeholders in Spanish-speaking countries to introduce GCR areas in national risk plans, as has been done with Argentina, and continuing with other partnerships with institutions such as ALLFED, Simon Institute, etc.
Seize the opportunity: Being GCR a neglected area, all the reports, articles, and other academic production in Spanish-speaking and with an emphasis on Latin America have a high probability of impact and reception, as we have verified in the development of our projects.
Betting on our staff training: Our team comes from different academic areas and various countries such as Spain, Mexico, and Colombia. We have different approaches that have served the development of projects with an interdisciplinary perspective and have facilitated engagement with stakeholders. This has been particularly important when engaging with stakeholders keen on seeing views from the Global South, such as the UN.
Evidence-Based Decision Making: With our publications and projects under development, we have detected that some areas are more promising in certain countries than others. We plan to use this knowledge to tailor our outreach strategies efficiently.

Our 95th-percentile version looks like

Integrate the most promising themes of our four main areas into the Spanish-speaking countries in which the best opportunities are detected or in the process of renewing national risk management plans.
Collaborations and partnerships flourish, resulting in an expansive network of allies who share our vision and contribute to our collective success.
We've achieved remarkable growth and successful financial stability that will allow us to continue our work long-term, improve the team's and interested people's training, and develop workshops in different countries to achieve significant stakeholder acceptance.
In this version, we have reached the Spanish-speaking community that works in risk management and the community, making them aware of the importance of longtermism and global catastrophic risks.

Indeed, Jaime and Juan actively participate in the staff mentoring process, along with our advisers. We also have developed internal training modules, which could become training material for internal and external mentoring. So far, our training has successfully taken our staff from having little exposure to Global Catastrophic Risk studies to actively contributing to high-quality reports in a couple of months. We have little capacity to expand and grow because Jaime and Juan's time is limited, but this will remain a pillar of our organization.

If we were to emphasize mentorship and movement-building as our primary goals, it could lead to significant transformations in how we operate. Here are some specific ways this could manifest:

Structured Mentorship Programs: Our ongoing staff training has resulted in well-structured training modules endorsed by experts. These modules, along with recommended reading materials and resources from EA communities, 80000 hours, and others, could serve as the foundation for a mentorship program.
Leadership Development and Talent Identification: Through previous recruitment processes, we've identified promising individuals within the Spanish-speaking effective altruism community. These individuals show a keen interest in advancing and leading in this field. Notably, our essay contest has been successful in spotting emerging talent.
Sharing Resources Platform: Our website boasts the largest collection of Spanish articles on global catastrophic risks, and we're committed to expanding this repository. This rich content will undoubtedly be a valuable resource for learners from various backgrounds.
Engaging the Community: We've successfully hosted events to introduce our reports and intend to continue doing so. Additionally, we're considering establishing platforms for talks at local universities and facilitating workshops for community participation.
Collaborative Initiatives: Collaborative projects have proven effective in enhancing staff training. Working with experts from organizations like ALLFED and EPOCH has resulted in products like the ASRS and AI reports. Furthermore, we've contributed to a United Nations policy brief in collaboration with the Simon Institute.

It's important to note that we already play a pivotal role within our community. As the sole organization dedicated to nurturing experts in global catastrophic risks within the Spanish-speaking community, our impact can have a cascading effect because other people learn about the topic through our reports and receive mentorship from the experts we are forming.

It's unclear whether we should prioritize more community building over other work. We're open to considering the idea of securing additional resources to expand our efforts and launch more mentorship-focused programs.

We are currently drafting a chapter on the ASRS for inclusion in Argentina's National Risk Reduction Plan for the period 2024-2030. Our objective is to contribute to more national plans across various Latin American countries, recognizing the unique opportunity to make a substantial impact. Given the region's geographical characteristics, Argentina and its neighboring countries hold pivotal roles in ensuring food security during scenarios such as nuclear winter and related scenarios.

Simultaneously, Spain is currently presiding over the European Council until the end of the year. We have established connections with an expert who will oversee discussions on regulating foundation models within the context of the EU AI Act in September. Additionally, we are in contact with a member of the European Parliament. Our intention is to maintain these connections and ensure that both individuals are well-informed about the latest standards in AI governance.

Shifting our focus to biological risks, we are working on a report for Guatemala. This aims to underscore the importance of monitoring emerging infectious diseases. The Latin American region holds significant potential for the emergence of epidemiological hotspots. Through this report, we hope to draw attention to the necessity of proactive measures in this regard.

Remain at your disposal for any further inquiries.

Joel Becker

over 2 years ago

@JTorrescelis thank you for your reply!

After reading, my core worry remains: constraints on Jaime and Juan's time make me nervous about the community-building benefits of RCG, and the catastrophic risk reduction projects are not compelling enough to make up for this.

I could imagine being excited about the community-building benefits even without Jaime and Juan putting lots of time into mentoring. This would probably look like hearing about collaborations beyond Epoch and ALLFED, and/or signs that other relatively senior staff were going to devote significant time to mentoring.

I am not sure what evidence would convince me that the catastrophic risk reduction benefits are competitive with other proposals. One necessary thing would be increasing the concreteness of your 95th percentile outcomes sketch. But, even then, I'm not sure that "significantly improving biological surveillance in Guatemala" would be a compelling enough result. (This is part of what leads to me emphasize community-building benefits.)

So, though I continue to be open to hearing more evidence and/or criticism of my view, I think I will not fund this project right now.

To be clear, I think this is really sad. I continue to be excited about:

Funding projects that other grantmakers do not have the time to get enough context on -- Spanish-speaking catastrophic risk work where I know some key organizers seems like a great example of this.
Spain/CDMX/LatAm as a non-US/UK hub for high-impact projects.
Any project that receives a large personal donation from Nuno Sempere.