June 30, 2014
We recently added the following question to our opinion pool:
Before January 1, 2015, will the leader of another major militia group in eastern Democratic Republic of Congo surrender?*
This might seem like an odd topic for a system designed to assess risks of future mass atrocities. As this question implies, however, we also plan to use the system from time to time to assess prospects for the cessation or reduction of atrocities in cases where they are already happening. Our open question on the possibility of a peace deal this year in Colombia is another example.
Even allowing this wider mission, though, our question about surrenders in eastern DRC might seem peculiarly narrow. If we want to know whether or not the atrocity-producing conflict in that region is trending in a better direction, why not just ask that directly?
The issue is what psychologist and forecasting expert Phil Tetlock calls the "rigor-relevance trade-off." Fuzzy questions about broad topics may be more relevant to our core concerns, but a lack of rigor in defining concepts and assessing outcomes often makes those questions less useful, too. As I wrote in a recent post on the Good Judgment Project's blog,
For the forecast’s consumers, we need to be able to explain clearly what a forecast does and does not cover, so they can use the information appropriately. As forecasters, we need to understand what we’re being asked to anticipate so we can think clearly about the forces and pathways that might or might not produce the relevant outcome. And then there’s the matter of scoring the results. If we can’t agree on what eventually happened, we won’t agree on the accuracy of the predictions. Then the consumers don’t know how reliable those forecasts are, the producers don’t get the feedback they need, and everyone gets frustrated and demotivated.
So, instead of asking directly about a salient but fuzzy topic, we look for events or other markers a) that we will know if and when we see them and b) whose occurrence or absence will, we believe, reveal something diagnostic about the deeper topic in which we're really interested.
In this instance, rather than asking our forecasters about "conflict" or "the peace process" in eastern DRC, we have asked them to assess the chances of a specific and potentially significant turn occurring in that process in the near future. The idea for this particular focal point came from Laura Seay, an assistant professor of political science at Colby College who is widely recognized as an expert on conflict in the DRC and, as it happens, is also a participant in our opinion pool.**
When I asked Laura for ideas about events that would bode well or ill for the DRC or the wider Great Lakes region, she identified additional rebel-movement surrenders as a critical issue. In 2013, the U.N. beefed up its mission to DRC in an attempt to shove the armed conflicts there toward a more definitive end. Earlier this year, the leader of one such movement surrendered himself to Congolese officials, only to be killed under ambiguous circumstances. That turn of events has raised concerns about the prospects for further surrenders and the effectiveness of the U.N.'s new strategy—concerns that the surrender of another militia leader would help to allay.
So far, our experts seem moderately pessimistic about the chances of additional leadership surrenders this year—and thus, implicitly, about the trajectory of peace-making efforts in eastern DRC. Only a handful of forecasters have weighed in since we posted the question, but you don't necessarily need a large crowd to generate a useful forecast. Right now, the consensus estimate is 34 percent, or about 2:1 odds against more surrenders in 2014. (If the website we're building were already up, I would put a link here to the page that provide live updates on the consensus forecast on this and all of our other opinion-pool questions. For now, though, this reported snapshot will have to do.)
---
* For purposes of the question, we specified that "only the following are considered major militia groups: Democratic Forces for the Liberation of Rwanda (FDLR), Raia Mutomboki, Allied Democratic Forces-National Army for the Liberation of Uganda (ADF-NALU), and Mai Mai Sheka. We also clarified for forecasters that "surrender" means "an individual described by one or more of those sources as the leader of one of the aforementioned militias must surrender himself to Congolese forces or authorities, and that "the killing or capture of a militia leader, or the surrender of militia members not described as a group’s leader," would not count as a "yes" for us.
** Laura assented to this acknowledgement. We do not publicly identify participants in our opinion pool without their permission.
View All Blog Posts