February 22, 2021
.jpg){kind=logo}
For the eighth year in a row, the Early Warning Project ran a comparison survey in December to solicit wisdom-of-the-crowd opinions on countries' relative risks for new mass killing.
Definition: Mass Killing
By our definition, a mass killing occurs when the deliberate actions of armed groups in a country (including but not limited to state security forces, rebel armies, and other militias) result in the deaths of at least 1,000 noncombatant civilians in that country over a period of one year or less. The civilians must also have been targeted for being part of a specific group.
Key Findings
According to participants, who voted on a series of head-to-head match-ups, the 30 countries at highest risk for a new mass killing in 2021 are:
- South Sudan
- Somalia
- Yemen
- Democratic Republic of Congo (Congo-Kinshasa)
- Syria
- Nigeria
- Ethiopia
- Sri Lanka
- Eritrea
- Libya
- Central African Republic
- Burundi
- Iraq
- Pakistan
- Afghanistan
- Sudan
- Burma/Myanmar
- Venezuela
- Burkina Faso
- Lebanon
- Egypt
- India
- Bangladesh
- Republic of the Congo (Congo-Brazzaville)
- Mali
- Philippines
- Mozambique
- Iran
- China
- Uganda
In the list above, countries in bold were experiencing mass killing as of the end of 2019, and most of these mass killings were likely ongoing through 2020 (EWP staff are currently assessing where new mass killings began in 2020).
This survey is part of our effort to experiment with wisdom-of-the-crowd methods for atrocity risk forecasting.
The comparison survey serves the same function as our annual Statistical Risk Assessment (SRA), which uses publicly available data and a machine learning model: to help focus additional resources where preventive action is most needed by ranking all countries in the world based on their risk for new mass killing.
Unlike our SRA, which has been tested extensively, we do not know how accurate results from the comparison survey are. Nevertheless, these survey results can be useful in two ways:
- Encouraging officials to focus policy attention on atrocity risks in countries that rate highly across multiple methods, and
- Prompting analysts to look more deeply into cases where the judgment of the "crowd" diverges from the statistical model and/or expert or organizational perceptions of risk.
Eight of the top 10 countries, according to this year’s survey, also ranked in the top 15 in our statistical risk assessment for 2020-21. These results should reinforce the importance of focusing on risks of mass killing in these countries, all of which are already experiencing high levels of violence against civilians: South Sudan, Somalia, Yemen, Democratic Republic of Congo (Congo-Kinshasa), Syria, Nigeria, and Ethiopia.
The survey results also draw attention to two countries that rank much lower on our statistical risk assessment: Sri Lanka (8th) and Eritrea (9th).
Survey respondents may have assessed Sri Lanka to be high-risk due to the country’s history of mass atrocities committed during its 26-year-long civil war, and in more recent years periods of political instability and large-scale terrorist attacks.
Respondents’ assessment of Eritrea could be informed by its alleged role in the ongoing conflict in the Tigray region of neighboring Ethiopia and alarms raised about the fate of Eritrean refugees in Ethiopia.
When assessing future risks in a country, analysts should consider the questions raised in the US Department of State/USAID Atrocity Assessment Framework:
- Which, if any, key actors currently have or might plausibly develop the motive, means, and opportunity to carry out large-scale, deliberate attacks on civilians?
- Which, if any, groups of civilians are currently being targeted or might plausibly be targeted for deliberate attack?
- Which other actors are enabling atrocities, or playing protecting or peacebuilding roles with respect to ongoing or potential mass atrocities?
- What grievances, if any, are driving an increase in atrocity risk? (e.g., history of past atrocities, lack of access to or competition for resources and/or land, identity group discrimination, lack of political representation, corruption, and impunity.)
- What resiliencies—social relationships, structures or processes that are able to provide dispute resolution and meet basic needs through non-violent means—are present?
- What are possible triggers of mass atrocities?
About the Survey
The annual comparison survey (formerly called “wiki survey”) is one of the quantitative methods used by the Early Warning Project to assess atrocity risk worldwide. The comparison survey methodology draws on a “wisdom of the crowd” approach. Our survey was open to the public for the month of December 2020, and specifically promoted to experts, policymakers, NGOs, and scholars in international affairs.
A pairwise comparison survey involves a single question—in this case, Which country is more likely to see a new episode of mass killing in 2021?—with many possible answers on which participants vote, one pair at a time. The response options presented to participants (all are countries with a population greater than 500,000) are randomly generated throughout the survey period.
The results from our annual comparison survey do not contribute directly to our statistical risk assessments, but they do inform the selection of cases we select for deep-dive qualitative assessments.
We asked participants to compare countries based on their risk for new mass killing. Any country could experience a new mass killing in 2021, whether or not one was ongoing at the end of 2020. For example, at the end of 2020 there was a nonstate-led mass killing ongoing in Central African Republic (CAR), but if the state intentionally targets a group of over 1000 civilians in 2021, that would qualify as a new mass killing. Though this is possible, it is statistically less likely to have multiple mass killings in a country at any given time. In short, survey respondents may have confused the risk of a new mass killing with the persistence or escalation of an ongoing one.
Methodology
We used All Our Ideas to conduct this pairwise comparison survey. For more on how a pairwise comparison survey works, see this working paper by the instrument’s creators. For details on what we mean by “mass killing” and “onset,” see the blog post that led respondents to the survey. We ask about mass killing risk because fatalities are, in general, more consistently reported across cases than other mass atrocity crimes.
We do not know the identities of our survey respondents. Participation in All Our Ideas surveys is anonymous, and our blog post was shared publicly. However, we do know that participants cast 4,062 votes in 54 unique user sessions. Some individuals likely voted in more than one session, so the total count of respondents is some unknown figure less than or equal to 54.
View All Blog Posts