Getting Both Shoes On

Our results show heavy usage of the donated TOMS shoes; indeed, the modal response was that children in the treatment group wore the shoes every day of the week. However, our study finds that the donated shoes did not significantly reduce shoelessness and had insignificant impacts in most of our key categories [health, school attendance] that would indicate transformative impacts.

argentina_shoes_blake-copyThis is from Bruce Wydick et al’s study of TOMS in El Salvador, released last month. The economists’ advanced statistical analysis confirms the long-held suspicions of most practitioners: TOMS have no “transformative impact” on children’s health or education.

Despite this, TOMS is by many measures the most successful social enterprise of all time. TOMS has given away more than 50 million shoes and spawned a host of copycat operations. My friend Justin put it:

“TOMS may have missed the point in doing effective social good, but they absolutely nailed effective branding. They understood the need to parse a complex issue into a simple message…that’s why their motto, “one for one,” is genius. In just three words you know exactly what they do and you have a cause to believe in”

This puzzles and frustrates me. Bringing communities out of poverty requires nuanced understanding and complicated intervention. However, raising money to bring communities out of poverty requires simple messages and relatable solutions. We need land reform and electoral accountability, we give goats and shoes.

Wydick and his team get this. They recognize that in-kind donations of food and clothes play a huge role in helping people in rich countries connect with the people they are donating to. They understand that despite the best theoretical knowledge, organizations have no choice but to follow the practices that allow them to keep operating.

So instead of lambasting TOMS for naive programming, the researchers offer a set of strategies for organizations to do better.

Like most scholars, they address the importance of foregrounding local needs and reducing dependency. For example, in El Salvador, the government already provides all schoolchildren with shoes. Moreover, the worm that causes the most foot infections has been eradicated in El Salvador.

Also, when physical goods are given out, Wydick and his team suggest that they should be given as rewards for positive activities like attending health checks. This builds a sense of accomplishment instead of dependency.

Finally, the authors introduced the idea of a “median impact narrative” as a contrast to the sort of ‘silver bullet’ stories so prevalent to the field. They suggest that instead of telling the story of an outlier, we tell the story of the average. So instead of a microloan spawning a multinational corporation, talk about the average impact that an intervention will have on each and every beneficiary. The authors of the paper created a statistically average ‘José Mantaro’ to describe TOMS’ impact:

“José Mantaro lives in San Francisco de Javier, El Salvador,…José’s family does not own a refrigerator, television, or radio, but both he and his brother have a bike…José walks 30 minutes to school, where he is in the third grade, but he has not yet learned to read or write.”

“José received a pair of TOMS shoes immediately after the baseline study in June 2012…He spent a little bit more time collecting water for his family and a little less time per day watching TV. An honest appraisal would suggest that receiving the shoes did not bring about transformative changes in Jose’s life…Yet the frequency with which he wore the new shoes indicates that the shoes donated by TOMS were nevertheless a welcome and appreciated gift.”

Of course, this reads a bit corny. It’s hard to imagine getting it by anyone in a marketing department. But the idea intrigues me- can we effectively pair statistics and stories? Perhaps for a more useful intervention, this kind of authenticity would be good marketing. I want to take part in development that effectively responds to real needs and inspires people to join the process. No answers yet, but I’m learning!

If you skip over the wild bootstrapping part, the paper is very readable, as is Wydick’s blog post on median impact narratives. I would recommend them both!