If not overhead, then what? Maybe this.

The theme of this blog can be summed up in a few sentences:There are thousands of nonprofit organizations in the US. Some do great work. Many do not. Wouldn't it be good to know which is which?Dean Karlan, an influential economist who teaches at Yale and founded an organization called Innovations for Poverty Action, last week unveiled a new venture that will reward better-performing nonprofits, albeit in a modest way and at a small scale at first. The new nonprofit, called ImpactMatters, has begun to do "impact audits"of charities. I wrote a story [paywall] about ImpactMatters last week for The Chronicle of Philanthropy.Evaluating charities is a tricky business. GiveWell (which I blogged about here) and Animal Charity Evaluators (here), two meta-charities that are part of the Effective Altruism movement, do the job very well. They do deep dives into a very small number of nonprofits and deliver value to people who care about global poverty or animal welfare.By contrast, Charity Navigator, America's biggest and most influential charity evaluator, ranks many more charities--about 8,000--but its focus remains transparency, governance and the maligned-but-persistent overhead ratio. (See my blogpost, Evaluating charities: If not overhead, then what?) Charity Navigator spotlight bad nonprofits, but it doesn't have the ability to identify good ones.So those donors who care about climate change, human rights, education inequality in the US or most other causes can't get much useful guidance, at least not when it comes to the actual work that nonprofits do."Donor ignorance is rampant," Karlan wrote in a brief Wall Street Journal essay announcing ImpactMatters.To address the problem, ImpactMatters has devised a plan to do "impact audits" of charities that are analogous to financial audits. The audits will focus on four elements: transparency, cost-effectiveness, cooperation with others and “theory of change.” Charities will pass or fail the audits; failed audits will be kept confidential to encourage more charities to participate. Sample audits of four charities -- the Success for All Foundation, the BOMA project, Trickle Up and Possible -- are posted on the ImpactMatters website.Karlan hopes that the audits will be paid for by donors or potential donors who care about making a difference.

“We are talking to parties who want to disrupt the philanthropy market,” he told me. 

Of course, every startup wants to disrupt something. That said, the philanthropy market, such as it is, desperately needs disruption. We'd all be better off if high-performing nonprofits captured a bigger share of the $350 billion or so that Americans give to charity each year.

If all goes well, ImpactMatters should help. At the very least, Karlan's track record makes this venture worth watching.

A 45-year-old development and behavioral economist, Karlan launched his first nonprofit, Innovations for Poverty Action (IPA), in 2002.With its sister organization, the Abdul Latif Jameel Poverty Action Lab (J-PAL) at MIT, IPA has  has helped change the way governments and nonprofits think about global poverty. They have also spread the use of randomized control trials to test anti-poverty programs and policies. IPA, which brought in more than $42 million in revenues last year, has over the years coordinated more than 500 studies with academics around the world. In other words, Karlan is an academic who gets things done. (He also co-founded a self-improvement website called stickK that, if nothing else, is disrupting the traditional spelling market.)

What's more, the directors of ImpactMatters include Paul Brest, the former president of the Hewlett Foundation;  Kevin Starr, managing director of the Mulago Foundation; and Michael Weinstein, chief program officer of the Robin Hood Foundation--all three experts in charity evaluation.

In a couple of conversations, Karlan stressed to me that ImpactMatters is just getting going. "We're going to start small, and learn," he said. He's inviting feedback.

I see at least three challenges ahead for ImpactMatters. First, the concept needs to scale. Until dozens or more likely hundreds of charities pass (or fail) audits, donors won't insist on them. For ImpactMatters to grow, Karlan may need a major foundation or a Silicon Valley billionaire who cares about effective philanthropy to write him a big check to get off the ground. Persuading donors to finance one audit at a time could be a slog.

Second, even if audits catch on, it's not clear that donors will pay them much heed. In surveys like Money for Good, donors say they  want their money to be well spent but only about one-third do any research into nonprofits and fewer than 10 percent compare charities. Much giving is done with the heart. Of course, that could be because there's a dearth of data. Hope Neighbor, who led the survey, recently told me: “We are essentially expecting donors to be like financial analysts, and have the acumen to answer all these questions about nonprofits." 

Finally, while we need better data to evaluate nonprofits, We need to keep in mind the risks that come with relying too heavily on metrics. Advocacy, for instance, is hard to measure. How can we measure the impact of 350.org on the climate change debate in the US? Or the work of small human rights groups around the world? Who's going to be disadvantaged, in other words, by audits?

This, of course, would be a good problem to have because it would mean that impact audits had become standard practice in the nonprofit world. It's a long way from here to there. But ImpactMatters looks to me like an important step in the right direction.