The ambition
The Tor Project (Tor), founded in Seattle in 2006, is a global non-profit organisation that creates and deploys free, open-source technologies and tools which help to advance human rights and digital freedoms around the world.
Tor is supported by the Swedish International Development Cooperation Agency (Sida) to advance its mission in the Global South. This financial backing allowed Tor to launch its Global South Strategy (GSS) in 2017 to connect more closely with at-risk communities such as journalists, LGBT+ groups and human rights activists.
The objectives of the strategy were twofold:
a) to build a community of Tor users in regions in the Global South in order to empower people to be more secure on the Internet,
b) to build a user-centric development process to facilitate usability improvements of Tor technologies and products.
The work included conducting outreach to target organisations, running digital security training sessions with human rights defenders, activists, and journalists, and conducting non-invasive user tests/assessments on Tor tools.
By mid-2022, the GSS had reached a level of maturity. Tor sought to conduct an independent and qualitative evaluation to understand the overall impact of the Global South Strategy, including its relevance to at-risk users and the role it plays to advance Tor’s mission.
Tor limited the evaluation to Colombia, Kenya and Uganda as core countries, but the strategy extends to several other countries in the Global South.
Our approach
Firetail attended the Tor Project’s all-hands meeting in Ireland to agree the project scope, explore definitions of success, and develop an evaluation framework.
This framework would help to structure the research design and shape the evaluation outputs around the key impact areas. Together with Tor’s commissioning team, we agreed on the following framework:
The impact
The findings and conclusions were synthesised into an evaluation report that found the GSS had been impactful and relevant across all four impact areas to some degree.
There is strong evidence of tangible examples of impact, such as partners knowing how to use Tor tools after training sessions or processes to support local translation. Anecdotes from the evaluation found evidence of a small yet active community, growing in both non-technical reach and inclusivity.
Firetail made recommendations to help improve the impact of the GSS in the future, including ways to formalise the strategy and be more strategic about communications and scaling up.
The Tor Project has reported the evaluation process and outcomes in a recent blog post. The evaluation has sparked bigger conversations among Tor teams around proactive community engagement, and the relevance and impact of programmes beyond just their duration.
Tor is now planning the next phase of the GSS and actively implementing the evaluation findings.
“The evaluation report from Firetail allowed us to understand the impact of our Global South Strategy not only at Tor but with our partners. It became a great tool for us to talk about this work internally and externally. It also allowed us to build our next three years of work to continue to grow our strategy and improve its impact. It was the first time that we had such an evaluation about our work so we were not sure what to expect, however the final result became an extremely important tool for us and we are very happy with it. There is nothing related to the process that I would do differently.”
Isabela Fernandes, Executive Director
Insights and further reading
Learn more about the Tor Project’s programmes and work or follow them on Twitter.
If you would like to learn more about our approach to evaluations, please get in touch.