At Treadwell, we are pursuing innovations in nonprofit program evaluation that prioritize equity, inclusion, and community leadership. Last year, Treadwell Consultant Jordan Grover worked with organizers in the DC area to engage youth in leading their own evaluations at three local nonprofits. She describes an exciting collaboration between nonprofit data professionals, youth participants, and students at American University in her article (below).
The initiative showed that nonprofits can improve their service delivery by engaging their participants in defining their own evaluation strategies and program metrics. The “Participatory Evaluation” process goes beyond surveying and interviewing program participants. It asks them to become leaders in designing research questions and indicators for how to measure the impact of the organizations that serve them.
Research for Change: Piloting Initiatives for Youth-Led Evaluation
As the drive for evidence-based programming influences national policy and funding, nonprofit organizations have been tasked with collecting more data and improving program metrics. Direct service organizations have team members dedicated to these metrics who are responsible for designing the program’s theory of change. They’re also responsible for planning and executing a plan for data collection, and analyzing the data to test for participant outcomes. One organization might choose to measure the impact of stable housing on a residents’ success in securing employment. Another might be interested in learning if after-school programming was successful at lowering absenteeism. Whoever defines the research question determines what data to collect, what analysis to conduct, and how to define program success. The metrics we choose outline the program goal posts and will directly impact service provision to program participants.
Creating Research For Change
Measure4Change, a community of practice for nonprofit data professionals, recognizes that there is power in who defines program metrics. Far too often, the participants of a program are left out of the program decision making or are only consulted on surveys or in focus groups. One strategy to involve participants in setting the program goal posts is to invite them to lead the evaluation. In the Spring of 2021, Jubilee Housing, Latin American Youth Center, and DC Scores invited youth in the DMV to define their own questions and set the stage for more youth-led evaluations across the region.
The youth were high school students and participants of the nonprofit organizations. They partnered with undergraduate students enrolled in a Community-Based Research class at American University to create the region’s first Research for Change initiative. Program staff members invited their youth participants to join Research for Change by attending Zoom calls with the students. The students were either Community-Based Research Scholars or certificate students at American University.
Youth participants learned the steps of a proper program evaluation so they could conduct an evaluation assessment alongside American University student mentors. Sessions lasted between March and May. The teens participated in weekly classes on research and evaluation, survey design and sampling, interview preparation, data collection, and analysis. American University students led all the classes. High school and college students worked collaboratively in small groups to define their own research questions, design surveys and interviews. Afterwards, they collected data from other youth at each partner organization.
Lessons Learned
The most significant challenge in executing this initiative was recruiting high school youth to join. Collaborators incentivized participants by offering community service hours but were unable to collect funds to offer stipends. The organization who had the most successful attendance was able to build time with American University students directly into their after-school programming. A second challenge was timeline. Because collaborators spent more time on recruitment, students only had 6 weeks to plan, collect data, and move to analysis.
Despite the challenges, this pilot initiative was a huge success. The members of Measure4Change would love to see it expand in the future, potentially with more funding. Students from across the DMV were able to attend classes online and collaborate on Zoom. The students’ findings encouraged partner organizations to implement program changes or pursue more ways to collect data in the future. High school participants also completed a feedback survey. They demonstrated an increased knowledge about evaluation, interest in evaluation careers, and overall satisfaction with participation.
Significance of Youth-Led Evaluation
At the end of the analysis, teams of teens and university students presented their findings back to the organizations. Staff members and organization leadership listened as the youth they serve described to them new findings about their programs. Additionally, the groups provided recommendations for program changes and metrics for the programs to consider in the future.
One reason to encourage youth-led evaluation in the DMV is to diversify the pipeline of who will become future researchers and evaluators in our communities. Because there is power in defining the research questions and program indicators, we strive to ensure our young community members have the tools to fill future roles in the growing field of nonprofit evaluation.
Need help with program evaluation?
Better data means greater impact.
Treadwell designs customized program evaluation plans to measure your organization’s impact. Schedule a free consultation to see how we make your organization – and your data – stronger.
