50: What are RCTs and how will they transform development aid? With Noam Angrist

The Shin Fujiyama Podcast | Social Entrepreneurship | Nonprofit Organizations | International Development Aid | NGOs

Today’s guest is Noam Angrist, the founder of Young 1ove, an NGO providing sex education to 35,000+ young people in Botswana. Sex-ed is a complicated issue, and over the decades it’s been hard to tell what worked and what didn’t. In Botswana, where 22% of the population has HIV, much of it hadn’t worked.

But when Noam used a tool from the scientific community, he could actually tell what interventions worked. Like a scientist, Noam discarded the interventions that didn’t work and focused on the ones that did. Soon, teenage pregnancy dropped by 28% in the communities he worked with, and he had the evidence to prove it. The tool that Noam used was the randomized control trial (RCT).

In this episode, Noam talks about his experience carrying out RCTs and discusses their limitations, challenges, and financial costs. He faced countless personal struggles along the way, like using his personal savings to fund the startup years, not having funding as launch day neared, government officials obstructing the program, and having to make 11-hour drives through the dirt roads of Africa.

When things got tough, Noam reminded himself to, “stay fiercely optimistic,” and “push through even when things are collapsing around you.”

Noam’s goal is to provide sex-ed to a million youth in southern Africa in the coming years. For his work, Noam Angrist has been named in the Forbes 30 Under 30 List for Social Entrepreneurship.

Noam Angrist's Reading List

• Nudge: Improving Decisions About Health, Wealth, and Happiness by Richard H. Thaler and Cass R. Sunstein
• Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya By Pascaline Dupas

Noam Angrist Show Notes

Noam Angrist studied math and economics at MIT

He took a class at MIT with Esther Duflo, one of the founders of The Abdul Latif Jameel Poverty Action Lab (J-PAL). The Poverty Action Lab popularized the Randomized Control Trials.

Randomized control trials have revealed that most interventions don’t work

Noam Angrist was influenced by an article he read in Esther Duflo’s class: Do Teenagers Respond to HIV Risk Information? Evidence from a Field Experiment in Kenya By Pascaline Dupas

Noam worked for the World Bank and J-PAL upon graduation and realized that many research studies produce papers but not programs

Noam did a project at the World Bank about Botswana. He went to the University of Botswana to better understand the situation.

The university students give free tuition and provides student stipends. But when stipends run out at the end of the month, they become sugar babies.

According to the study by Dupas, the unprotected sex with sugar daddies was infecting many young people with HIV, but the research paper was not being turned into a program

Noam went out into the field with a team and turned the research into an actual program

Doers need plans that are simple, clear, and actionable but research papers are usually complicated and long.

The topics of sex and sugar babies are taboo

Noam and his team organized sex-ed workshops with the college students and found out that what the facilitators did or said affected the results greatly

Small, five-minute ice breakers at the beginning of the session changes the whole dynamic of the workshop and people talk more. Or when someone contributes a comment, everyone snaps and creates an environment of positivity and enthusiasm

The workshops are more effective when the facilitator is also a youth. It is truly a youth-to-youth model

Young 1ove now focus on girls ages 12-16

Women in areas with high HIV rates tend to have multiple, concurrent sexual partners

Due to the nature of the HIV virus, the disease is easier spread when less time has passed between sexual intercourse with different partners. That is why having concurrent partners is so dangerous.

45% of 40-year-old men in Botswana have HIV. 5% of young people have HIV. So the inter-generational sexual partnerships is infecting the younger generation.

Noam Angrist won $20,000 from D-Prize, a foundation that funds new entrepreneurs who increase access to proven poverty interventions.

Young 1ove carried out a second program (that had been proven to work in Kenya ten years prior) in Botswana, funded by J-PAL, Baylor HIV Clinic, and the Ministry of Education

Noam wanted to know if what worked in Kenya would work in Botswana

Noam thought he would become a development economics professor before the opportunity to run Young 1ove had came up

He had been offered a full ride to go to graduate school in the US but instead became a social entrepreneur

“Why do research when you can use it to make a difference?”

There was a gap... the good research was not being used

“It’s not the lack of good research. That exists. It’s the using and adapting of it that interested me.”

J-PAL and World Bank have hundreds of studies showing what works and what doesn’t, but the studies are not being used enough

NGOs should ask themselves, what do you need to do an RCT for? Is it to demonstrate impact to get more funding or is it to understand if an intervention is actually working to adjust or scale your programs?

If you want to do the RCT to give you more credibility and better branding, then you should hire an expensive, credible third party like J-PAL to do it for you

But if you want to do the RCT just as a learning tool for your own organization’s internal purposes, you don’t need to spend much money

Currently, kids are being told to abstain, which has not been very helpful.

Simply encouraging young people to date other young people leads to safer sex

Mechanism matters

90% of girls think younger men have a higher HIV rate than older men. But when you reveal the truth, it changes the cost-benefit-analysis for the girls

Botswana is really good about distributing HIV medication. So now the cost of having HIV is lower. So the messaging for sex-ed has to evolve with the changes of cost-benefit-analysis of the situation

A lot of universities now have centers that perform RCT.

Reliable organizations that do RCTs: J-PAL, Innovations for Poverty Action (IPA), World Bank, ID Insight, The Busara Center for Behavioral Economics in Nairobi, Kenya

You should build into your organization’s fiber to always look to improve and constantly iterate

The original studies in Botswana produced ambiguous results. They measured more factors than the study from Kenya and had three groups instead of two that they studied

They were not ready to scale at that point

They tried different strategies, like instead of focusing on talking about the risk of HIV (now that the cost of having it is less), they experimented by focusing on the risk of pregnancy. Or by providing follow-up meetings to remind people

There is the temptation to do more and more research and not take a stance/position or move forward until one is 100% sure. But we are never 100% sure!

There is less evidence as to what works across countries and over time

MTV, The Ministry of Youth, and The Global Innovation Fund started to help

They delivered their programs to one third of the country in forty days

Young 1ove’s commitment to evidence-based interventions has attracted many donors

Most of the original staff members were volunteers

They applied for funding from Evidence Action and many other groups to find funding

Noam Angrist used his personal savings to withdraw cash out of ATMs in Botswana to fund the beginnings of Young 1ove

Everything was bootstrapped in the beginning. They operated lean, and shared computers at the university

During one 11-hour drive, Noam received a phone call from J-PAL about funding for a quarter million dollar RCT. They awarded the money and Noam was ecstatic and called his staff immediately. They received the funding one month before everything needed to start

They needed to get permission from the Ministries of the Botswana government. When ministers advocating for Young 1ove change jobs, it can complicate matters

They learned to get buy-in from EVERYONE in the government and so called and updated the stakeholders so much that they were told not to call so much

The marginal cost of changing what the workshop facilitators say is very low. The benefits could be huge.

Young 1ove is now working on a remedial education program

One of the best ways to prevent HIV is to keep kids in school

Noam describes his team as fiercely optimistic

“Push through even when things are collapsing around you. Stay fiercely optimistic.”

“You should be able to do good and do well.”

Salaries for nonprofit workers should be competitive

“What a pleasure it is wake up and want to go to work.”

The documentary, Broken for Good talks about the poor salaries in the nonprofit industry

Big Bang Philanthropy is a foundation that funds big but meddles/dictate little. They trust the organizations to make the right decisions. Getting donors like that is key.

“Stay crisp and simple, but internally brace for a lot more complexity. Juggle that tension and be aware of it.”