Water taps and information gaps

If you’ve donated money to a water charity, congratulations. You’ve stepped up to try to solve one of the world’s most pressing problems--the fact that roughly 750 million people do not have access to clean water.Has your donation made a lasting difference? That’s hard to know.Big water charities point to numbers that, they say, demonstrate their impact. Since its founding in 2006, charity: water says it has funded 16,138 water projects. Water.org, in its latest annual report, says that in 2013 it completed 174 community-based water projects, constructed 73,081 toilets, established 66,632 household water connections and served 606,012 people with water and sanitation. In 2013-2014, Water Aid says it reached 2 million people with water and 3 million with sanitation.But the charities, as a rule, do not report on how many of those projects are providing clean water a year, two or five years after they were built. Some don’t know. That, by itself, is a sign that something’s amiss.As it happens, the poor countries where these charities operate are littered with water projects that need repair. In what is admittedly an old study (2009), the International Institute for Environment and Development said there are 50,000 broken rural water points across Africa that represents a failed investment of US $215-360 million. Newer data on failure rates for water projects can be found here. Up to 40 percent of  rural water systems fail prematurely, and less than one percent of water, sanitation and hygiene projects have long-term monitoring, according to US AID.That’s unacceptable.This isn’t a new problem. Here’s Ned Breslin, the CEO of nonprofit Water for People, in a 2010 essay called Rethinking Hydro-Philanthropy:

Africa, Asia and Latin America have become wastelands for broken water and sanitation infrastructure….the general public is shielded from these hard truths as perceptions of failure could threaten “the cause” of reaching the unserved.The images that dominate the sector—pictures of children happily gulping water from a new tap or the counter-image of women collecting water from dirty puddles—do not tell the whole story. The real image should be the one that plays itself out every day all over the world of the woman walking slowly past a broken hand pump, bucket at her side or on her head, on her way to (or from) that scoop hole or dirty puddle that she once hoped would never again be part of her life.A new culture of accountability and transparency that transcends the nonsense that currently masquerades as reporting in the sector must emerge if poor people worldwide are to truly emerge from the drudgery of water collection.

Strong words. The good news is that the big water charities acknowledge this problem. Some formed a network called Sustainable WASH to focus on the durability of water, sanitation and hygiene projects. Others are using digital technology for monitoring.  All are taking meaningful steps to track the sustainability of their projects.Christophe Gorder, the chief global water officer of charity: water, says:

The pace & scale at which we’re collecting data on water points is unparalleled in the WASH industry. We still have a long way to go in terms of organizing, analyzing, understanding and acting on all this information. But, we are very committed and making good progress.

As a newcomer to the sector, I’ve begun to learn about water NGOs from reading their websites and talking to experts. It’s a complicated business, to say the least. While almost all the NGO websites display pictures like this one of smiling children enjoying clean water, they deploy different strategies, work in different countries, and focus on different markets, i.e., rural, urban or peri-urban (a fancy word for places next to a city). Some like charity : water are predominantly facilitators or middlemen: They raise money and distribute it to local partners who develop and build projects. Others like Water for People (which gets big donations from charity: water) are vertically integrated, raising money in richer countries and employing staff in the global south to build and manage projects. Water.org is best known for its Water Credit program, which provides access to small loans (average: $194) to people and entrepreneurs to spend or invest in water or sanitation projects, typically water taps or toilets. Whenever possible, Water Aid works with functioning local governments that are willing to insure that users will receive water and sanitation services over time.Here’s what I’ve learned, so far:

Water charities are growing fast

Globally, Water Aid, the biggest of the charities, which is headquartered in London, raised £81.8 million (roughly $123 million) in 2013-14; five years earlier, it raised £43.8 million ($65.8 million). In 2013, charity: water raised $35.9 million, up from $6.2 million in 2008. During that same five-year span, revenues at water.org more than doubled to $12.4 million from $5.5 million (when the organization was known as WaterPartners). While some gains can be attributed to the economic recovery, water is undoubtedly a hot sector.Give charity: water credit for its fundraising savvy. (This is an organization with a 63-page brand book. Really.) Its founder, Scott Harrison, a darling of Silicon Valley, has been profiled in The New York Times magazine, Fast Company and Wired. It's easy for supporters to start their own campaigns -- more than 64,000 have done so, many at birthday parties -- and it promises that it will “send 100% of what you raise directly to the field.” Private donors and foundations cover staff salaries and overhead; this is an appealing pitch, although it reinforces negative stereotypes about overhead.Water.org has Matt Damon, who by all accounts has invested his time and smarts in the nonprofit. As if that weren’t enough, corporation foundations love the microfinance model. Donors include the foundations of PepsiCo, Mastercard, Bank of America, Caterpillar and IKEA, according to Inside Philanthropy. Nevertheless, it’s important to recognize that charity can’t possibly solve the global water crisis. “For about $30 a person, we know how to help millions,” says charity: water. At that rate, providing clean safe water to 750 million people will cost $22.5 billion. 

It’s hard to evaluate water NGOs.

GiveWell, a organization I admire, investigated water charities back in 2011 and was unable to “prioritize” any of them as a top charity. GiveWell is careful to note that this does not constitute a negative ranking of the charities; some declined to provide the very detailed information that GiveWell sought, and others were “de-prioritized” for reasons that remain unclear. In 2013, GiveWell did a thorough review of research into water quality innovations like filtration and chlorination to see how effective they are at preventing waterborne diseases, particularly diarrhea. It found that “in development world settings where diarrhea is endemic, because of a lack of sanitation, hygiene and access to safe water, the evidence for the effectiveness of these interventions is unclear.”Sustainable WASH has encouraged water NGOs to sign its charter (many have done so) to complete a self-assessment (fewer have done so, they’re listed here) and to conduct third-party assessments of their sustainability (a work in progress, none have done so). Building off the work of a small foundation called the Fontes Foundation, Sustainable WASH is supporting the development of a metric called Water Person Years that would measure the number of people served over time. “That changes the whole incentive structure,” says Brian Banks of the Global Water Challenge and Sustainable WASH.Naively hoping to get some reliable numbers myself, I asked Water Aid, charity: water, Water.org and Water for People how many projects they installed in 2010 and whether they are working, five years later.None of the four charities had begun comprehensive monitoring in 2010, so I couldn’t get answers, although all provided selected data by email.At charity: water, Christophe Gorder said that during Q1 of 2015 the group visited 1,089 sites that were two to eight years old, and found that 87 percent were still working. Water.org said that a third-party evaluation of about 9,132 loans, most in India, had found that “97 percent of the water improvements were found to be in working condition.” I asked to see the report and was told it wasn’t public.Vincent Casey, a program manager with Water Aid, told me that “a study of four countries (one from each region we work in) looking back five years found that in the case of water supply, 76% of 2560 water points installed (serving approximately 307,000 people) were still working. Around 24% were down but of these, 40% were due to be fixed the following week.”Water for People has the most extensive reporting--no surprise since Ned Breslin has been outspoken about sustainability. It devotes a platform called Re-Imagine Reporting to detailed, country-by-country results of its monitoring, including financial data. Its monitors use a software platform called FLOW that was developed by Water for People but, because of significant demand from others, was transferred to a Dutch nonprofit called Akvo, which makes it available to numerous NGOs, foundations, governments and companies. Mark Duey, a Water for People executive, to me by phone that monitoring and evaluation “sounds boring. It’s not sexy, But it’s really critical.”The results of these surveys are not comparable. Nor does the industry have a good benchmark of what constitutes success, as Vincent Casey of Water Aid explains:

There is not enough data from enough places with sufficiently robust benchmarks to cross compare. Data from one district where the local government is well capacitated and where there is a diverse economy generating sufficient revenue to keep services running may show higher percentages. Data from a poor, remote, marginalised district with weak local government capacity and a limited cash economy may show lower percentages. Comparing the two without taking the background context into account would be like comparing apples with pears.

So I asked a couple of smart people...

In theory, could a donor or group of donors - or even a smart grad student, with time  -- identify the most effective water NGO?

Here’s how David Zetland, an academic, a blogger at Aguanomics and author of a new book called Living with Water Scarcity, responded:

Well, you'd have to run a randomized trial, "giving" water to half the communities and leaving the other half to figure out their own solutions.Then you'd want to compare health and wealth outcomes after a few years, to see if the charities had impact.If I had a grad student on this for a few years, I would have them compare baselines for ALL charity-assisted villages and THEN compare those after a few years to see how the charities ranked in their relative outcomes. That's feasible, in the same way as it's feasible to compare survival rates from heart surgery across hospitals or university graduation rates, BUT you need to try to compare for the baseline. Some charities may be better at picking good villages. Others may be better at building capacity. The first should not get as much credit as the second.Bottom line: it's possible and feasible but it takes time and effort and... "program evaluation" is VERY under-pursued for obvious reasons (don't look a gift horse in the mouth, even if it's dying...)

Brian Banks of the Global Water Challenge said:

The difficulty comes not only from data availability (which we’re working to address through the Water Point Data Exchange), but in the question itself. Effectiveness is an incredibly difficult to define...You have water treatment initiatives, pipes and taps projects, so called “software” or capacity building projects, projects that incorporate more or less hygiene, and some that involve environmental protection....Unfortunately, the sector still hasn’t come around a single definition of effectiveness, or even a single goal for water projects (i.e. economic empowerment, health improvement, educational outcomes, etc.). In this way, a highly effective project that focused extensively on capacity building may directly impact fewer people (i.e. higher cost per water person year), but be effective in improving the sector at large. There just isn’t a great way to compare apples to apples, as every project has so much nuance, especially when considering aspects such as reliability, affordability and quality.In short, while there is merit in exploring this, it has proven to be exceedingly difficult to compare different organizations against a common standard of effectiveness.

Well, that’s discouraging, but…

The water charities are getting better at monitoring.

You’re going to have to trust me on this, because this post is already long. But all of the water charities understand that monitoring is vital, not merely to appease and attract donors, but to improve.As Water Aid’s Vincent Casey put it: “It’s about NGOs being transparent, and learning from one another what’s working and what’s not working.”Technology will play a key role.At charity: water, there’s excitement over a $5 million project, launched at the end of 2012 and funded by Google, that has developed remote sensor technology that will monitor whether water is flowing at any of its projects, at any time, anywhere in the world.“Really the quantum leap is going to come with sensors,” Gorder told me. “We’re just a couple of months away from deploying thousands of these. The level of accountability will be huge.”To pay for repairs, when needed, charity: water has launched a new monthly subscription program called Pipeline -- although in an ideal world, those costs would either be built into the up-front cost of the water project or, better, paid for by fees collected from users. Gorder says that sustainability and monitoring are built into the project costs but “we commit to our donors that we’ll prove to them where every dollar was spent, so we want to put an end-date on when we spend their money, so we can report back to them, Typically, this is a 21-month process.”Donors want feel-good stories. There’s not much emotion in seeing an inspector with a clipboard checking out a five-year-old project….even if that would be the wisest use of donor funds.

So is the glass half-full? Or half-empty? 

As often happens when I explore new arenas, I’ve emerged from my shallow dive into the water sector with more questions than answers. This time around, that’s partly because the water charities have a ways to go when it comes to transparency.There’s no doubt that they could do better. A small, Seattle-based water NGO called Splash that values monitoring and evaluation maintains a website called Proving It that is model of transparency and accountability.The Global Water Challenge, meantime, is developing common metrics, through an initiative called the Water Point Data Exchange that will be launched next month. Brian Banks, who's part of the effort, tells me: "In a sector where sustainability wasn't even a mainstream consideration a decade ago, we've seen inspired progress."Still, there's lots of work to be done. Institutional donors that support the water NGOs could invest a percentage of their grant money into initiatives to evaluate and improve the entire sector. We could use the water equivalent of Animal Charities Evaluator.At the very least, each water charity could set clear goals and timetables. (“Everyone deserves the right to clean, safe, publicly-accessible water” is not a time-based goal.).They could better explain what they do. (They work on sanitation, but you don’t often see pictures of toilets on their websites, for obvious reasons.) They could report on their mistakes, and how they learn from them. (It’s possible.)Other questions on my mind: How can governments be shamed/encouraged to deliver clean safe water? What role can the private sector play? Why do more poor people have cell phones than water? And, when I do another post about water nonprofits, what should my focus be?