Fwd: Givewell's philosophy, and DCP2 errors
- The email below contains a few questions from Jacob Pekarek and GiveWell's answers. Thanks to Jacob for giving us permission to post!Best,AlexanderOn Wed, Nov 2, 2011 at 6:19 PM, Jacob Pekarek <jpekarek@...> wrote:
I would be interested to learn more about your analysis of errors in the DCP2 report on deworming, examine your position on fistulas and NTD's, and inquire about Givewell's philosophy of preventing suffering. I am a student at Trinity University, and am interested in Givewell's evaluations for the purposes of my own giving, and for the end of researching the most effective NGO's working in the third world. I am familiar with the philosophies and work of Givewell-- as to my questions on fistulas, I have read an earlier Givewell email exchange between Elie, and Jeremy, a teacher interested in donating to charities working towards the removal of obstetric fistula. I shall expand on these questions below.
First, I am generally having trouble understanding how I should interpret the "Givewell Cost Effectiveness Analysis" on the DCP2 deworming report. Specifically, the values of "prop b" within the "ascariasis" and "a.3" tabs are different-- I realize that this may be a result of a correction, but I would like to ask where I can reference the data which necessitates this change. I am still struggling to explain to myself how this flowchart disproves the DCP2 cost effectiveness efforts for deworming interventions. Nevertheless, my questions on Givewell's philosophy are of more importance than my questioning of how to make sense of the DCP2 data and errors.The "prop B" cells change as a result of a correction. The rationale is explained in our blog post:The figure in the “A/B” column refers number of people at risk for a given symptom, not the number of people suffering from that symptom. These are equivalent for Type A and Type C symptoms, but not for Type B symptoms including CIDTA. Intestinal Nematode Infections (PDF), the working paper that contains Table 9, says that “in any annual cohort of heavily infected children some 5% suffer [Type B symptoms, which are the only symptoms that have life-long effects]” (p. 26). Using the figures as the official calculation did would therefore lead to a 20x overstatement in the prevalence of CIDTA.The basic idea is that the 'ascariasis' (and 'trichuriasis' and 'hookworm') tab(s) assume that the same number of people have symptom B as have symptom A, but that's the result of a misreading of the source table.This mistake applies not just to cognitive impairment due to ascariasis, but also to cognitive impairment due to trichuriasis and hookworms, similarly leading to a 20x overstatement of the prevalence of cognitive impairment due to those infections as well.To your more general point that this flowchart doesn't disprove the DCP2 calculation, you're definitely right. The spreadsheet shows the corrections to the calculation, but does not explain or justify them. The blog post is the place to go for that.
Secondly, the presence of errors within the report on deworming leads me to question if the analysis of simpler, health based interventions could yield more meaningful results than the analysis of harder to measure interventions such as vaccinations, or deworming schemes. In other words, should one's prior be adjusted to reflect not only one's knowledge of organizations which specialize in a given intervention, but the tendency of analyses of these organizations (or interventions) to be flawed?My question is this: "Does the ease of evaluating the impact of interventions with clear results mean that these evaluations should be given greater consideration over evaluations of interventions with less clear results?" Specifically, do interventions to provide surgeries which remove obstetric fistulas deserve more consideration, relative to deworming interventions, or interventions which provide vaccination against NTD's? I, for one, think that guessing the past prevalence of a soil- transmitted helminth in any region, once mass school based deworming programs have been administered, would be a challenge; successful fistula surgeries seem easier to track than successfully dewormed patients, in my opinion.I think that the basic answer to your question is yes, with caveats. I don't think we should totally discount morbidities or interventions that are difficult to measure, but quality of evaluation is a definite consideration in favor of some causes. I don't think, though, that this is an especially good reason to prefer fistula operations to vaccines. Because most of the diseases that vaccines fight kill people (especially children), they are remarkably well-quantified and understood relative to many other interventions. Although the effects of vaccines are more dispersed than a fistula surgery, I don't think that they have less "clear results." As Holden has written, confidence in your evidence is definitely something that should affect your considered view of the cost-effectiveness of a charity.
Granted, there are many uncertainties and unseen costs which may apply to both interventions dealing with fistulas, and those dealing with vaccinations or deworming. The adjustments for the weight which different impairments carry (in terms of DALY's) are even a matter of contention. Still, I wonder if the (lack of?) ability of those whose fistulas are removed to re-involve themselves in village life is considered in the DALY handicap for fistula patients.
Lastly, I have one question on Givewell's philosophy of the prevention of suffering. Is the preferred outcome of the prevention of suffering to uplift those who are suffering in a way that enhances their social mobility, or to ease the suffering of those who suffer the most? (This is assuming that either option would result in a roughly equal net reduction of suffering).I don't think we have a definite view. We try to avoid answering philosophical questions until we reach a point where we need to in order to make charitable decisions, and we haven't yet on this question. That said, I think this blog post might be of interest.
Thank you for taking the time to read this extensive email, and my thanks also for the work you have done in researching and scrutinizing the charities which do the most to prevent suffering in the third world.
Best regards,Jacob Pekarek--Alexander BergerResearch Analyst