Advertising for good

[youtube https://www.youtube.com/watch?v=8uPzSy1yAX8]Advertising changes behavior. Why else would brands pay as much as $5 million for a 30-second spot on the Super Bowl? But measuring the impact of advertising, particularly in traditional media, is an inexact science at best. ("Half the money I spend on advertising is wasted," retailer John Wanamaker famously said. "I just don't know which half.") This uncertainty is one reason why advertising has not been more widely deployed to do good.Roy Head, the founder and chief executive of a London-based nonprofit called Development Media International, has set out to change that, by demonstrating that advertising can improve lives of people in poor countries. DMI runs radio, TV and mobile phone campaigns aimed at changing behaviors, most notably in Burkina Faso, where the NGO recently completed a sophisticated four-year randomized controlled trial (RCT) to test the hypothesis that radio ads can save the lives of children under five--perhaps at a lower cost than other interventionsUsing selected local radio stations, DMI flooded the airwaves with commercials (like the one above) urging mothers to breast feed their babies, to seek health care for children who exhibit symptoms of disease and to practice proper hygiene, among other things. It then tried to measure the impact of the project, which DMI calls "the most rigorous trial ever conducted of a media health intervention."So did the campaign save lives? Child mortality rates fell sharply, but it's hard to know how much of the improvement was caused by the radio ads. The lack of clarity underscores the challenges of trying to tease out cause and effect in real-world settings where social and political "noise" can drown out the "signal" of any experiment."It's so bloody difficult to do an RCT," lamented Head, when we spoke recently via Skype.I'm writing about DMI because it has been identified as a "standout charity" by GiveWell, which rigorously evaluates nonprofits. ("Standout charities" sit a notch below the four "top charities" recommended by GiveWell.) DMI is also a favorite nonprofit of The Life You Can Save, an organization devoted to effective altruism that I blogged about last year. DMI has earned widespread praise for its transparency and for its devotion to measuring impact. And its work has enormous potential. “Mass media could actually be cheaper than any other methods of saving people’s lives,” Head says.A former BBC TV producer, Head started DMI in 2005. Previously, he had set up radio and TV stations for the UN in Cambodia and the former Yugoslavia, and created media campaigns for the World Health Organization. DMI's earliest work was funded by the Joseph Roundtree Charitable Trust, a Quaker trust, which gave Head a "visionaries" grant to research behavior-change campaigns.Working with the London School of Hygiene and Tropical Medicine,  DMI developed a mathematical model that estimated how many children's lives could be saved by large-scale media campaigns targeting the key causes of under-five mortality. The DMI/LSHTM model forecast that campaigns could reduce child mortality in many low-income countries by 16% to 23%, depending on the profile of the country, at a cost per life-year saved lower than other interventions. Their research was published in The Lancet in February 2015.Here's how DMI explains its approach:

Our objective is behavior change. In recent years, the vast majority of aid money has been spent on improving the 'supply side' (doctors, hospitals, teachers, schools). All of this is absolutely necessary, but in many countries this investment has been at the expense of something equally important: creating demand (empowering communities with knowledge and encouraging positive behaviors). We run media campaigns that give people information, but also convince them to change their behaviors, overcoming the practical and cultural barriers in their way.

The trouble is, as DMI acknowledges: "There is little evidence that media campaigns are effective at changing behaviors in developing countries." Without compelling evidence, persuading governments or foundations to devote substantial sums of money to advertising -- instead of, say, building health clinics or distributing bed nets to prevent malaria -- is an uphill battle.So Head has been trying to test DMI's hypothesis."I want to do work that is scientific and rigorous," he told me. "How many people can we stop from being ill and at what cost? What's worth spending money on? What generates the best results?" He is doing all of this with modest resources: DMI spent about $3 million last year, less than the expense of one of those Super Bowl ads.Burkina Faso was selected for its first RCT because the landlocked, West African nation has a very localized, radio-dominated media landscape. That enables DMI to use local FM radio stations to broadcast messages to some regions without those messages "leaking" into others. Those who were exposed to the campaign can then be compared to a control group. Enormous care was devoted to developing the most effective messages, insuring they were broadcast as planned and tracking their impact, across a broad range of metrics.And yet, in the immortal words of John Lennon, "life is what happens to you while you’re busy making other plans." During the randomized control trial, the Bill & Melinda Gates Foundation and other funders supported a large training program for Burkinabé health workers that could have affected their clients and, thus, child mortality rates. Population Media Center, a US-based NGO that has been producing social-marketing campaigns for decades, conducted a campaign to promote family planning that might have had spillover effects. A national bed net distribution campaign was underway. Finally, in the midst of the trial, Burkina Faso's longtime president fled the country and a short-lived military coup took place, briefly affecting radio broadcasts. Still, DMI has declared at least a partial victory:

This is the first randomised controlled trial to demonstrate that mass media can cause behaviour change.

Even better, child mortality rates declined dramatically in Burkina Faso, from an estimated rate of 168 deaths per 1,000 to 96 deaths per 1,000 over a five year period. This is wonderful news--but it makes it hard to determine how much of a difference, if any, was made by DMI's campaign.GiveWell is skeptical. Its lengthy analysis of the RCT raises a slew of doubts about the study, cited here. After reviewing the latest evidence, GiveWell says it "did not find any effect of DMI's program on child mortality, and it found substantially less effect on behavior change" than earlier, midterm results. DMI has yet to release the final results and says it is doing more research, with the support of independent scientific advisors. There's disagreement, too, about how to measure the programs cost-per-life-saved, a key metric.The messiness of these results might be considered disappointing, given the time, money and effort put into the experiment by DMI.But this is how science works. More important, it's how global aid should work. You hypothesize. You experiment. You learn. You adapt, and try again.GiveWell continues to praise DMI for its "strong transparency, rigorous self-evaluation, and its work on a potentially cost-effective program." Other donors, including the Wellcome Trust, the Planet Wheeler Foundation (from the creators of the Lonely Planet guidebooks), the Global Innovation Fund (a project of five government development agencies and the Omidyar Network), the Mulago Foundation and the Gates Foundation, have stepped in to support DMI as it runs campaigns around family planning and child mortality in the Democratic Republic of the Congo and begins a second RCT, around family planning, in Burkina Faso.Uncertainty, after all, hasn't stopped Coke, Pepsi and McDonald's from spending small fortunes on marketing, much of it in the world's poorest countries. It shouldn't stand in the way of advertising for good--so long as the work is carried out by organizations, like DMI, that are open and willing to learn.